Image filter device

ABSTRACT

A loop filter ( 16 ) includes a filter processing unit ( 162   c ) configured to generate an output image by acting on an input image made up of a plurality of unit regions, and a reference region setting unit ( 162   b ) configured to reduce a reference region to be referenced by the filter processing unit ( 162   c ) for calculating a pixel value of each target pixel to a size according to a position of this target pixel in a unit region including this target pixel.

TECHNICAL FIELD

The present invention relates to an image filter device configured toperform filtering of an image, also relates to an encoding device and adecoding device which include such an image filter device, and alsorelates to a data structure of encoded data to be decoded by such adecoding device.

BACKGROUND ART

In order to effectively transmit or record a moving image, there havebeen employed a moving image encoding device (encoding device)configured to generate encoded data by encoding the moving image, and amoving image decoding device (decoding device) configured to generate adecoded image by decoding this encoded data. Examples of a specificmoving image coding system include H.264/MPEG-4.AVC (NPL 1), a systememployed by KTA software which is codec for joint development in VCEG(Video Coding Expert Group), and a system (NPL 2) to be employed by TMuC(Test Model under Consideration) software which is a succeeding codec.

With such a coding system, an image (picture) making up a moving imageis managed by a layered structure made up of slices to be obtained bydividing an image, the maximum coding increment (LCU: Largest CodingUnit) to be obtained by dividing a slice, and coding increment (CU:Coding Unit) to be obtained by dividing a largest coding unit, a blockand a partition to be obtained by dividing a coding unit, and in manycases, encoding is performed with a block as the minimum unit.

Also, with such a coding system, in general, a predicted image isgenerated based on a local decoded image to be obtained byencoding/decoding an input image, and a difference image (may alsoreferred to as residual image or predicted residual) between thispredicted image and an input image is encoded. Also, as a method forgenerating a predicted image, there has been known a method calledinter-frame prediction (inter prediction) and intra-screen prediction(intra prediction).

With inter prediction, motion compensation using a motion vector isapplied to a reference image within a reference frame (decoded image) ofwhich the entire frame has been decoded, and accordingly, a predictedimage within a prediction target frame is generated for each predictionunit. On the other hand, with intra prediction, based on a local decodedimage within the same frame, a predicted image in this frame issequentially generated.

With NPL 2, an adaptive loop filter (ALF: Adaptive Loop Filter)(hereinafter, also simply referred to as “adaptive filter”) configuredto perform filter processing on a decoded image for each coding unit hasbeen disclosed. This adaptive filter determines a filter coefficientwhere error between a decoded image subjected to filtering and theoriginal image becomes the minimum, for each slice on a decoded image,and subjects each coding unit included in this slice to filtering basedon this filter coefficient. An encoding device and a decoding devicewhich include such an adaptive filter can improve precision of apredicted image to be generated by referencing an image subjected tofiltering by improving image quality of an encoded/decoded image usingfiltering, and accordingly, coding efficiency can be improved.

Also, the adaptive filter disclosed in NPL 2 can realize increase inspeed of filter processing by performing filter processing withreference to a pixel value transferred to local memory which is localmemory in which a target coding unit which is a processing target and animage in the vicinity thereof can be stored which can be accessed athigh speed from frame memory of which the access speed is low which isframe memory in which pixel values of the entire frame can be stored.

CITATION LIST Non Patent Literature

-   NPL 1: “Recommendation ITU-T H.264”, Telecommunication    Standardization Sector of ITU, March 2009 (disclosed in March, 2009)-   NPL 2: “Test Model under Consideration JCTVC-B205 draft007”, Joint    Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and    ISO/IEC JTC1/SC29WG11, 2nd Meeting: Geneva, CH, July 2010 (disclosed    in July, 2010)

SUMMARY OF INVENTION Technical Problem

However, an adaptive filter according to the related art disclosed inNPL 2 references a pixel value other than a target coding unit (a pixelvalue of a pixel in the vicinity of a target coding unit) in addition toa pixel value within a target coding unit at the time of performingfilter processing on a target coding unit, and accordingly, this causesa problem wherein the amount of data to be transferred from the framememory to the local memory increases. Description will specifically bemade regarding this point with reference to FIG. 29.

FIG. 29 illustrates, with an adaptive filter according to the relatedart, in the event that the size of a target coding unit CU is 8×8pixels, a reference region R with five taps to be set according to aposition of a target pixel, and a reference block RA which is asummation group of the reference region R regarding each target pixelincluded in a target coding unit. The reference block RA is a group ofpixels to be referenced by the adaptive filter according to the relatedart for generating an image subjected to filtering regarding a targetcoding unit. In the case of the example in FIG. 29, the adaptive filteraccording to the related art has to reference each pixel value includedin the reference block RA with 12×12 pixels for generating an imagesubjected to filtering regarding a target coding unit with 8×8 pixels.In this manner, the adaptive filter according to the related artreferences, at the time of performing filter processing regarding atarget coding unit, a pixel value in the vicinity of a target codingunit in addition to a pixel value within a target coding unit, andaccordingly, this causes a problem wherein the amount of data to betransferred from the frame memory to the local memory increases.

On the other hand, in order to reduce the amount of data to betransferred, for example, it can be conceived to decrease the size of areference region, but in this case, prediction precision of a predictedimage to be calculated with an image subjected to filtering decreases,and accordingly, this causes a problem wherein coding efficiencydecreases.

The present invention has been made in the light of the above problems,and its object is to realize an image filter device whereby throughputand processing time can be reduced by reducing the amount of data to betransferred at the time of adaptive filter processing while maintaininghigh coding efficiency.

Solution to Problem

In order to solve the above-mentioned problems, an image filter deviceaccording to the present invention including: filter means configured togenerate an output image by acting on an input image made up of aplurality of unit regions; and reference region modifying meansconfigured to reduce a reference region to be referenced for the filtermeans calculating a pixel value of each target pixel to a size accordingto a position of this target pixel in a unit region including thistarget pixel.

In the event that the image filter device is employed for a decodingdevice configured to decode encoded data to generate a decoded image, oran encoding device configured to encode an image to be encoded togenerate encoded data, a predicted image with high prediction precisioncan be generated with reference to an output image of the image filterdevice, and accordingly, coding efficiency is improved.

Also, according to the image filter device configured as describedabove, a reference region to be referenced for the filter meanscalculating a pixel value of each target pixel is reduced to a sizeaccording to a position of this target pixel in a unit region includingthis target pixel, and accordingly, the number of pixels to bereferenced for generating an output image can be reduced whilemaintaining high coding efficiency. Accordingly, according to the imagefilter device configured as described above, the amount of data to betransferred at the time of filter processing can be reduced whilemaintaining high coding efficiency.

Note that the unit region mentioned here may be, for example, thelargest coding unit LCU, or may be a coding unit CU to be obtained bydividing the largest coding unit LCU, or may be another region.

Also, an image filter device according to the present inventionincluding: filter means configured to generate an output image for eachunit region by acting on an input image made up of a plurality of unitregions; and smoothing means configured to smooth a pixel value of apixel where of the unit regions, distance from at least one of two edgeson the upstream side in processing order is equal to or shorter than DB;with the filter means acting on a target unit region after the smoothingmeans act on two edges on the upstream side of a target unit regionbefore acting on two edges on the upstream side of the next unit regionin processing order.

Also, an image filter device according to the present inventionincluding: smoothing means configured to smooth a pixel value of a pixelpositioned near at least one of two edges on the upstream side inprocessing order of each unit region regarding a decoded image to beobtained by adding a predicted image to be generated for each unitregion and a residual image; calculating means including first filtermeans configured to act on an output image from the smoothing means,second filter means configured to act on the predicted image, and thirdfilter means configured to act on the residual image, configured to addand output output images from the first to third filter means; andreference region setting means configured to set a reference regionwhich the first to third filter means reference for calculating a pixelvalue of each target pixel, and to set the number of pixels in thevertical direction of a reference region to 1.

According to the image filter device, an output image is generated withreference to a decoded image, predicted image, and a residual imagesubjected to smoothing processing by the smoothing means. Accordingly, adecoding device including the image filter device, and an encodingdevice including the image filter device generate a predicted image withreference to an output image of the image filter device, andaccordingly, coding efficiency is improved. Also, the number of pixelsin the vertical direction of the reference region is set to 1, andaccordingly, the throughput of filter processing and the amount of datato be transferred are reduced.

Also, an image filter device according to the present inventionincluding: filter means configured to generate an output image by actingon an input image made up of one or a plurality of slices; filter meansconfigured to calculate a pixel value of a target pixel in the outputimage from each pixel value of the input image in a reference region setin the vicinity of this target pixel; and target pixel restricting meansconfigured to restrict a position of a target pixel to a position wherethe entire reference region including this target pixel is included in aslice including this target pixel.

According to the image filter device, an output image in a target sliceis generated with reference to a pixel value alone within a targetslice. In other words, an output image in a target slice is generatedwithout referencing a pixel outside the target slice. Accordingly,according to the above-mentioned configuration, throughput forgenerating an output image is reduced. Also, according to theconfiguration, filter processing can be performed without awaiting untileach pixel in the next slice in processing order which is a sliceadjacent to a target slice can be referenced, and accordingly,processing time is reduced.

Advantageous Effects of Invention

As described above, a filter device according to the present inventionincludes: filter means configured to generate an output image by actingon an input image made up of a plurality of unit regions; and referenceregion modifying means configured to reduce a reference region to bereferenced for the filter means calculating a pixel value of each targetpixel to a size according to a position of this target pixel in a unitregion including this target pixel.

According to the image filter device configured as described above, theamount of data to be transferred at the time of filter processing can bereduced while maintaining high coding efficiency.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an adaptivefilter including a moving image decoding device according to a firstembodiment of the present invention.

FIG. 2 is a diagram illustrating a data structure of encoded data to begenerated by the moving image encoding device according to the firstembodiment of the present invention and to be referenced by the movingimage decoding device according to the first embodiment of the presentinvention. (a) illustrates a configuration of a picture layer of encodeddata, (b) illustrates a configuration of a slice layer included in thepicture layer, (c) illustrates a configuration of each CU making up anLCU layer included in the slice layer, (d) illustrates a configurationof a leaf CU included in the CU layer, (e) illustrates a configurationof inter prediction information regarding the leaf CU, (f) illustrates aconfiguration of intra prediction information regarding the leaf CU, and(g) illustrates a configuration of a filter parameter included in aslice header.

FIG. 3 is a block diagram illustrating a configuration of the movingimage decoding device according to the first embodiment of the presentinvention.

FIG. 4 is a diagram for describing deblocking processing by a deblockingfilter which a loop filter according to the first embodiment of thepresent invention has, (a) illustrates processing order for multipleLCUs included in a slice, (b) illustrates processing order for multipleCUs included in the LCU, (c) illustrates a border (thick solid line) ofdeblocking completion at a point-in-time when deblocking regarding anLCU made up of multiple CUs is completed, and a border (thin solid line)of deblocking incompletion.

FIG. 5 is a diagram for describing processing by a loop filter accordingto the first embodiment of the present invention, (a) illustrates afirst filter processing unit ALFU1 to be set regarding a target LCU in afirst processing example, (b) illustrates a flowchart of the firstprocessing example, (c) illustrates a second filter processing unitALFU2 to be set regarding a target LCU in a second processing example,and (b) illustrates a flowchart of the second processing example.

FIG. 6 is a diagram for describing processing by an adaptive filterincluding a loop filter according to the first embodiment of the presentinvention, (a) illustrates an example of a reference region to be set bythe adaptive filter, and (b) illustrates another example of a referenceregion to be set by the adaptive filter.

FIG. 7 is a flowchart illustrating a flow of processing for generating adecoded image subjected to adaptive filtering by the adaptive filteraccording to the first embodiment of the present invention.

FIG. 8 is a diagram for describing processing for setting a referenceregion by the adaptive filter according to the first embodiment of thepresent invention, and (a) to (e) are flowcharts illustrating a flow ofeach setting example.

FIG. 9 is a diagram exemplifying a reference region to be set by theadaptive filter according to the first embodiment of the presentinvention.

FIG. 10 is a diagram for describing processing by the adaptive filteraccording to the first embodiment of the present invention, and is adiagram illustrating a position relation between a target LCU and afilter processing unit ALFU corresponding to the target LCU.

FIG. 11 is a diagram for describing processing by the adaptive filteraccording to the first embodiment of the present invention, (a)illustrates a target pixel within a filter processing unit ALFU, and (b)illustrates a reference region to be set in the vicinity of a targetpixel.

FIG. 12 is a diagram for describing a reduction advantage of the amountof data to be transferred by the adaptive filter according to the firstembodiment of the present invention, (a) illustrates a region of a pixelto be transferred at adaptive filter processing according to the relatedart, and (b) to (d) illustrate a region of a pixel to be transferred ina first processing example according to the present invention.

FIG. 13 is a diagram for describing a reduction advantage of the amountof data to be transferred by the adaptive filter according to the firstembodiment of the present invention, (a) illustrates a region of a pixelto be transferred at adaptive filter processing according to the relatedart, and (b) illustrates a region of a pixel to be transferred in asecond processing example according to the present invention.

FIG. 14 is a diagram for describing processing by the adaptive filteraccording to the first embodiment of the present invention, and (a) to(d) are diagrams for describing each constraint to be imposed at thetime of setting a reference region.

FIG. 15 is for describing processing by the adaptive filter according tothe first embodiment of the present invention, and is a tableillustrating a specific example of an intra-ALFU Y coordinate, anintra-LCU Y coordinate, a normalized intra-LCU Y coordinate, an upwardreferable range, a first downward referable range, and a second downwardreferable range.

FIG. 16 is for describing processing by the adaptive filter according tothe first embodiment of the present invention, and is a tableillustrating a specific example of an intra-LCU Y coordinate, an upwardreferable range, and a first downward referable range.

FIG. 17 is for describing processing by the adaptive filter according tothe first embodiment of the present invention, and is a tableillustrating a specific example of an intra-ALFU Y coordinate, anintra-LCU Y coordinate, a normalized intra-LCU Y coordinate, an upwardreferable range, a first downward referable range, and a second downwardreferable range.

FIG. 18 is for describing processing by the adaptive filter according tothe first embodiment of the present invention, and is a tableillustrating a specific example of a normalized intra-LCU Y coordinate,an upward referable range, a first downward referable range, and asecond downward referable range.

FIG. 19 is a diagram illustrating an example of a reference region to beset by the adaptive filter according to the first embodiment of thepresent invention.

FIG. 20 is a block diagram illustrating a configuration of a movingimage encoding device according to the first embodiment of the presentinvention.

FIG. 21 is a block diagram illustrating a configuration of a loop filterincluding the moving image encoding device according to the firstembodiment of the present invention.

FIG. 22 is a block diagram illustrating a configuration of a movingimage decoding device according to a second embodiment of the presentinvention.

FIG. 23 is a block diagram illustrating a configuration of a loop filterincluding the moving image decoding device according to the secondembodiment of the present invention.

FIG. 24 is a block diagram illustrating a configuration of a movingimage encoding device according to the second embodiment of the presentinvention.

FIG. 25 is a block diagram illustrating a configuration of a loop filterincluding the moving image encoding device according to the secondembodiment of the present invention.

FIG. 26 is a block diagram illustrating a configuration of a loop filteraccording to a third embodiment of the present invention.

FIG. 27 is a diagram for describing processing by the loop filteraccording to the third embodiment of the present invention, (a)illustrates an internal region and an adjacent region to be set on aslice by this loop filter, and (b) illustrates a region to be referencedby a loop filter according to the related art.

FIG. 28 is a diagram illustrating an example of a reference region to beset by an adaptive filter according to a fourth embodiment of thepresent invention.

FIG. 29 is a diagram illustrating a reference region R to be setaccording to a position of a target pixel, and a reference block RAwhich is a summation group of the reference region R regarding eachtarget pixel included in a target coding unit, with a conventionaladaptive filter.

FIG. 30 is a diagram for describing that a moving image decoding deviceand a moving image encoding device according to an embodiment of thepresent invention can be used for transmission and reception of a movingimage, (a) is a block diagram illustrating a configuration of atransmission device on which the moving image encoding device has beenmounted, and (b) is a block diagram illustrating a configuration of areception device on which the moving image decoding device has beenmounted.

FIG. 31 is a diagram for describing that a moving image decoding deviceand a moving image encoding device according to an embodiment of thepresent invention can be used for recording and playing of a movingimage, (a) is a block diagram illustrating a configuration of arecording device on which the moving image encoding device has beenmounted, and (b) is a block diagram illustrating a configuration of aplaying device on which the moving image decoding device has beenmounted.

DESCRIPTION OF EMBODIMENTS

Description regarding an embodiment of a decoding device and an encodingdevice according to the present invention based on drawings is asfollows. Note that the decoding device according to the presentembodiment is for decoding a moving image from encoded data.Accordingly, hereinafter, this will be referred to as “moving imagedecoding device”. Also, the encoding device according to the presentembodiment is for generating encoded data by encoding a moving image.Accordingly, hereinafter, this will be referred to as “moving imageencoding device”.

However, the application range of the present invention is notrestricted to this. Specifically, as apparent from the followingdescription, features of the present invention are realized withoutassuming multiple frames. That is to say, the present invention can beapplied to decoding devices in general and encoding devices in generalregardless of whether to take a moving image as a target or a stillimage as a target.

(Configuration of Encoded Data #1)

Prior to description of a moving image decoding device 1 according tothe present embodiment, a structure of encoded data #1 to be generatedby a moving image encoding device 2 according to the present embodimentand to be decoded by the moving image decoding device 1 will bedescribed with reference to FIG. 2. The encoded data #1 includes alayered structure made up of a sequence layer, a GOP (Group Of Picture)layer, a picture layer, a slice layer, and a largest coding unit (LCU:Largest Coding Unit) layer.

A structure of layers of the picture layer and thereafter in the encodeddata #1 is illustrated in FIG. 2. (a) to (f) in FIG. 2 are diagramsillustrating a structure of a picture layer P, a slice layer S, an LCUlayer LCU, a leaf CU (referred to as CUL in (d) in FIG. 2) included inthe LCU, inter prediction information PI_Inter which is predictioninformation PI regarding an inter prediction (inter-frame prediction)partition, and intra prediction information PI_Intra which is predictioninformation PI regarding an intra prediction (intra-screen prediction)partition, respectively.

(Picture Layer)

The picture layer P is a group of data to be referenced by the movingimage decoding device 1 for decoding a target picture which is a pictureto be processed. The picture layer P includes, as illustrated in (a) inFIG. 2, a picture header PH, and slice layers S1 to SNs (Ns is a totalnumber of slice layers included in the picture layer P).

The picture header PH includes an encoding parameter group to bereferenced by the moving image decoding device 1 for determining adecoding method of a target picture. For example, encoding modeinformation (entropy_coding_mode_flag) which indicates a mode ofvariable length coding to be used by the moving image encoding device 2at the time of encoding is an example of encoding parameters included inthe picture header PH.

(Slice Layer)

The slice layers S included in the picture layer P are a group of datato be referenced by the moving image decoding device 1 for decoding atarget slice which is a slice to be processed. The slice layer Sincludes, as illustrated in (b) in FIG. 2, a slice header SH, and LCUlayers LCU1 to LCUNc (Nc is a total number of LCUs included in the sliceS).

The slice header SH includes a encoding parameter group to be referencedby the moving image decoding device 1 for determining a decoding methodof a target slice. Slice type specifying information (slice_type) forspecifying a slice type is an example of the encoding parametersincluded in the slice header SH.

Examples of a slice type that can be specified with the slice typespecifying information include (1) I slice using intra prediction aloneat the time of encoding, (2) single directional prediction at the timeof encoding, or P slice using intra prediction, and (3) uni predictionor bi prediction at the time of encoding, or B slice using intraprediction.

Also, the slice header SH includes filter parameters FP to be referencedby the adaptive filter included in the moving image decoding device 1.Note that a configuration of the filter parameters FP will be describedlater, and accordingly, description thereof will be omitted here.

(Lcu Layer)

The LCU layers LCU included in the slice layer S are a group of data tobe referenced by the moving image decoding device 1 for decoding atarget LCU which is an LCU to be processed.

The LCU layer LCU is made up of multiple coding units (CU: Coding unit)to be obtained by hierarchically dividing this LCU into quadtreedivision. In other words, the LCU layer LCU is, of a layered structurerecursively including multiple CUs, a coding unit equivalent to theuppermost. The CUs included in the LCU layer LCU has, as illustrated in(c) in FIG. 2, a CU header CUH, and a layered structure recursivelyincluding multiple CUs to be obtained by quadtree division of this CU.

The size of each CU excluding its LCU is a vertically and horizontallyhalf of the size of a CU to which this CU directly belongs (i.e., CU ofone hierarchy higher order of this CU), and the size that each CU cantake depends on the size and hierarchical depth of the LCU included in asequence parameter set SPS of the encoded data #1. For example, in theevent that the size of the LCU is 128×128 pixels, and the maximumhierarchical depth is 5, CUs in hierarchies equal to or lower than thisLCU can take five types of sizes, that is, any of 128×128 pixels, 64×64pixels, 32×32 pixels, 16×16 pixels, and 8×8 pixels. Also, a CU which isnot further divided will be referred to as a leaf CU.

(CU Header)

The CU header CUH includes an encoding parameter to be referenced by themoving image decoding device 1 for determining a method for decoding atarget CU. Specifically, as illustrated in (c) in FIG. 2, there isincluded a CU division flag SP_CU for specifying whether to furtherdivide the target CU four ways to lower-order. In the event that the CUdivision flag SP_CU is 0, that is, in the event that the CU is notfurther divided, this CU is a leaf CU.

(Leaf CU)

The CU that is not further divided (leaf of CU) is treated as aprediction unit (PU: Prediction Unit) and a transform unit (TU:Transform Unit).

As illustrated in (d) in FIG. 2, a leaf CU (referred to as CUL in (d) inFIG. 2) includes (1) PU information PUI to be referenced at the time ofa predicted image being generated by the moving image decoding device 1,and (2) TU information TUI to be referenced at the time of residual databeing decoded by the moving image decoding device 1.

A skip flag SKIP is a flag that indicates whether or not a skip mode isapplied to a target PU, and in the event that the value of the skip flagSKIP is 1, that is, in the event that the skip mode has been applied tothe target leaf, the PU information PUI and TU information TUI in theleaf CU thereof are omitted. Note that the skip flag SKIP is omitted inthe I slice.

The PU information PUI includes, as illustrated in (d) in FIG. 2, theskip flag SKIP, prediction type information PT, and predictioninformation PI. The prediction type information PT is information forspecifying whether to use intra prediction or inter prediction as aprediction image generating method regard a target leaf CU (target PU).The prediction information PI is made up of intra prediction informationPI_Intra or inter prediction information PI_Inter according to whichprediction procedure the prediction type information PT specifies.Hereinafter, a PU to which intra prediction is applied will also bereferred to as an intra PU, and a PU to which inter prediction isapplied will also be referred to as an inter PU.

The PU information PUI includes the shape, size, and informationspecifying a position within a target PU, of each partition included ina target PU. The partition mentioned here is a single or multiple notoverlapped regions which make up a target leaf CU, and generation of apredicted image is performed with a partition as a unit.

The TU information TUI includes, as illustrated in (d) in FIG. 2, aquantization parameter difference Δqp (tu_qp_delta) for specifying thesize of a quantization step, TU division information SP_TU forspecifying a division pattern for each block of a target leaf CU (targetTU), and quantized predicted residual QD1 to QDNT (NT is a total numberof blocks included in the target TU).

The quantization parameter difference Δqp is difference qp−qp′ between aquantization parameter qp in the target TU, and a quantization parameterqp′ in a TU encoded immediately before the TU thereof.

The TU division information SP_TU is specifically the shape, size, andinformation specifying a position within the target TU of each blockincluded in the target TU. Each TU can take a size from 64×64 pixels to2×2 pixels, for example. The block mentioned here is a single ormultiple not overlapped regions which make up the target leaf CU, andencoding/decoding of predicted residual is performed with a block as aunit.

Each quantized predicted residual QD is encoded data generated by themoving image encoding device 2 subjecting a target block which is ablock to be processed to the following processes 1 to 3. Process 1: Apredicted residual obtained by subtracting a predicted image from animage to be encoded is subjected to DCT transform (Discrete CosineTransform). Process 2: A DCT coefficient obtained in Process 1 issubjected to quantization. Process 3: The DCT coefficient subjected toquantization in Process 2 is subjected to variable length encoding. Theabove-mentioned quantization parameter qp represents the size of aquantization step QP used at the time of the moving image encodingdevice 2 subjecting the DCT coefficient to quantization (QP=2^(qp/6)).

(Inter Prediction Information PI_Inter)

The inter prediction information PI_Inter includes encoding parametersto be referenced at the time of the moving image decoding device 1generating an inter predicted image by inter prediction. As illustratedin (e) in FIG. 2, the inter prediction information PI_Inter includesinter PU division information SP_Inter for specifying a division patternfor each partition of the target PU, and inter prediction parametersPP_Inter1 to PP_InterNe (Ne is a total number of inter predictionpartitions included in the target PU) regarding the partitions.

The inter PU division information SP_Inter is specifically the shape,size, and information specifying a position within the target PU of eachinter prediction partition included in the target PU (inter PU).

The inter PU can be divided into eight types of partitions in total offour symmetric divisions (symmetric splittings) of 2N×2N pixels, 2N×Npixels, N×2N pixels, and N×N pixels, and four asymmetric divisions(asymmetric splittings) of 2N×nU pixels, 2N×nD pixels, nL×2N pixels, andnR×2N pixels. A specific value of N is stipulated with the size of a CUto which this PU belongs, and specific values of nU, nD, nL, and nR aredetermined according to the value of N. For example, the inter PU of128×128 pixels can be divided into inter prediction partitions of128×128 pixels, 128×64 pixels, 64×128 pixels, 64×64 pixels, 128×32pixels, 128×96 pixels, 32×128 pixels, or 96×128 pixels.

(Inter Prediction Parameters PP_Inter)

The inter prediction parameters PP_Inter include, as illustrated in (e)in FIG. 2, a reference image index RI, an estimated motion vector indexPMVI, and a motion vector residual MVD.

The motion vector residual MVD is encoded data generated by the movingimage encoding device 2 executing the following processes 4 to 6.Process 4: An encoded/decoded local decoded image (more accurately,image obtained by subjecting an encoded/decoded local decoded image todeblocking and adaptive filter processing), and a motion vector my for atarget partition is derived with reference to the selectedencoded/decoded local decoded image (hereinafter, also referred to as“reference image”). Process 5: An estimating method is selected, anestimated value (hereinafter, also referred to as “estimated motionvector”) pmv of the motion vector my to be assigned to the targetpartition is derived using the selected estimating method. Process 6:The motion vector residual MVD obtained by subtracting the estimatedmotion vector pmv derived in Process 5 from the motion vector my derivedin Process 4 is encoded.

The reference image index RI is for specifying an encoded/decoded localdecoded image (reference image) selected in Process 4, and theabove-mentioned estimated motion vector index PMVI is for specifying theestimating method selected in Process 5. Examples of the estimatingmethod that can be selected in Process 5 include (1) With a localdecoded image being encoded/decoded (more accurately, image to beobtained by subjecting a decoded region of a local decoded image beingencoded/decoded to deblocking and adaptive filter processing), a methodwherein median of a motion vector assigned to a partition adjacent tothe target partition (hereinafter, also referred to as “adjacentpartition”) is taken as the estimated motion vector pmv, and (2) With anencoded/decoded local decoded image, a method wherein a motion vectorassigned to a partition which occupies the same position as with thetarget partition (often referred to as “colocate partition”) is taken asthe estimated motion vector pmv.

Note that prediction parameters PP regarding a partition where uniprediction is performed include, as illustrated in (e) in FIG. 2, eachof the reference image index RI, estimated motion vector index PMVI, andmotion vector residual MVD one at a time, but prediction parameters PPregarding a partition where bi prediction (weighted prediction) isperformed include two reference image indexes RI1 and RI2, two estimatedmotion vector indexes PMVI1 and PMVI2, and two motion vector residualsMVD1 and MVD2.

(Intra Prediction Information PI_Intra)

The intra prediction information PI_Intra includes encoding parametersto be referenced at the time of the moving image decoding device 1generating an intra predicted image by intra prediction. As illustratedin (f) in FIG. 2, the intra prediction information PI_Intra includesintra PU division information SP_Intra for specifying a division patternfor the partitions of the target PU (intra PU), and intra predictionparameters PP_Intra1 to PP_IntraNa (Na is a total number of intraprediction partitions included in the target PU) regarding thepartitions.

The intra PU division information SP_Intra is specifically the shape,size, and information specifying a position within the target PU of eachintra prediction partition included in the target PU. The intra PUdivision information SP_Intra includes an intra division flag(intra_split_flag) for specifying whether to divide the target PU intopartitions. In the event that the intra division flag is 1, the targetPU is symmetrically divided into four partitions, and in the event thatthe intra division flag is 0, the target PU is treated as one partitionwithout being divided. Accordingly, if we say that the size of thetarget PU is 2N×2N pixels, the intra prediction partition obtains anysize of 2N×2N pixels (without division) and N×N pixels (four divisions)(here, N=2^(n), n is an optional integer equal to or greater than 1).For example, an intra PU of 128×128 pixels can be divided into intraprediction partitions of 128×128 pixels and 64×64 pixels.

(Intra Prediction Parameters PP_Intra)

The intra prediction parameters PP_Intra include, as illustrated in (f)in FIG. 2, an estimation flag MPM, and a residual prediction mode indexRIPM. The intra prediction parameters PP_Intra are parameters forspecifying an intra prediction procedure (prediction mode) regarding thepartitions.

The estimation flag MPM is a flag that indicates whether a predictionmode estimated based on a prediction mode assigned to a partition in thevicinity of the target partition which is a processing target agreeswith the prediction mode regarding this target partition. Examples ofthe partition in the vicinity of the target partition include apartition adjacent to the upper edge of the target partition, and apartition adjacent to the left edge of the target partition.

The residual prediction mode index RIPM is an index to be included inthe intra prediction parameters PP_Intra in the event that the estimatedprediction mode differs from the prediction mode regarding the targetpartition, and is an index for specifying a prediction mode to beassigned to this target partition.

(Filter Parameters FP)

As described above, the slice header SH includes the filter parametersFP to be referenced by the adaptive filter included in the moving imagedecoding device 1. The filter parameters FP include, as illustrated in(g) in FIG. 2, a filter coefficient group. The filter coefficient groupincludes (1) tap count specification information for specifying thenumber of taps of the filter, (2) filter coefficients a₀ to a_(NT-1) (NTis a total number of filter coefficients included in the filtercoefficient group), and (3) offset o.

(Moving Image Decoding Device 1)

Hereinafter, the moving image decoding device 1 according to the presentembodiment will be described with reference to FIG. 1 to FIG. 19. Themoving image decoding device 1 is a decoding device which includesH.264/MPEG-4.AVC, technology employed by KTA software which is a codecfor joint development in VCEG (Video Coding Expert Group), andtechnology employed by TMuC (Test Model under Consideration) softwarewhich is a succeeding codec thereof at a portion thereof.

FIG. 3 is a block diagram illustrating the configuration of the movingimage decoding device 1. As illustrated in FIG. 3, the moving imagedecoding device 1 includes a variable length code decoding unit 11, apredicted image generator 12, an inverse quantization/inversetransformation unit 13, an adder 14, frame memory 15, and a loop filter16. Also, as illustrated in FIG. 3, the predicted image generator 12includes a motion vector restoring unit 12 a, an inter predicted imagegenerator 12 b, an intra predicted image generator 12 c, and aprediction procedure determining unit 12 d. The moving image decodingdevice 1 is a device configured to generate a moving image #2 bydecoding encoded data #1.

(Variable Length Code Decoding Unit 11)

The variable length code decoding unit 11 decodes prediction parametersPP regarding the partitions from the encoded data #1 to supply these tothe predicted image generator 12. Specifically, the variable length codedecoding unit 11 decodes, with regard to an inter prediction partition,the inter prediction parameters PP_Inter including the reference imageindex RI, estimated motion vector index PMVI, and motion vector residualMVD from the encoded data #1, and supplies these to the motion vectorrestoring unit 12 a. On the other hand, the variable length codedecoding unit 11 decodes, with regard to an intra prediction partition,the intra prediction parameters PP_Intra including the estimation flagMPM, residual index RIPM, and additional index AI from the encoded data#1, and supplies these to the intra predicted image generator 12 c.

Also, the variable length code decoding unit 11 decodes the predictiontype information PT regarding each of the partitions from the encodeddata #1, and supplies this to the prediction procedure determining unit12 d. Further, the variable length code decoding unit 11 decodes thequantization prediction residual QD regarding each of the blocks, andthe quantization parameter difference Δqp regarding the TU including theblock thereof from the encoded data #1, and supplies these to theinverse quantization/inverse transformation unit 13. Also, the variablelength code decoding unit 11 decodes the filter parameters FP from theencoded data #1, and supplies this to the loop filter 16.

(Predicted Image Generator 12)

The predicted image generator 12 identifies whether each partition is aninter prediction partition to be subjected to inter prediction or anintra prediction partition to be subjected to intra prediction based onthe prediction type information PT regarding each partition. In the caseof the former, the predicted image generator 12 generates an interpredicted image Pred_Inter, and also supplies the generated interpredicted image Pred_Inter to the adder 14 as a predicted image Pred,and in the case of the latter, generates an intra predicted imagePred_Intra, and also supplies the generated intra predicted imagePred_Intra to the adder 14. Note that, in the event that the skip modehas been applied to the PU to be processed, the predicted imagegenerator 12 omits decoding of another parameter belonging to this PU.

(Motion Vector Restoring Unit 12 a)

The motion vector restoring unit 12 a restores the motion vector myregarding each inter prediction partition from the motion vectorresidual MVD regarding the partition thereof, and a decoded motionvector my′ regarding another partition. Specifically, (1) In accordancewith an estimation method specified by the estimated motion vector indexPMVI, the estimated motion vector pmv is derived from the decoded motionvector my′, and (2) The motion vector my is obtained by adding thederived estimated motion vector pmv and motion vector residual MVD. Notethat the decoded motion vector my′ regarding another partition can beread out from the frame memory 15. The motion vector restoring unit 12 asupplies the restored motion vector my to the inter predicted imagegenerator 12 b along with the corresponding reference image index RI.

(Inter Predicted Image Generator 12 b)

The inter predicted image generator 12 b generates a motion compensationimage mc regarding each inter prediction partition by inter-frameprediction. Specifically, the inter predicted image generator 12 bgenerates a motion compensation image mc from an adaptive-filtereddecoded image P_ALF′ specified by the reference image index RI suppliedfrom the same motion vector restoring unit 12 a using the motion vectormy supplied form the motion vector restoring unit 12 a. Here, theadaptive-filtered decoded image P_ALF′ is an image to be obtained bysubjecting a decoded image where the entire frame has already beencompleted to filter processing by the loop filter 16, and the interpredicted image generator 12 b can read out the pixel value of eachpixel which makes up the adaptive-filtered decoded image P_ALF′ from theframe memory 15. The motion compensation image mc generated by the interpredicted image generator 12 b is supplied to the prediction proceduredetermining unit 12 d as the inter predicted image Pred_Inter.

(Intra Predicted Image Generator 12 c)

The intra predicted image generator 12 c generates a predicted imagePred_Intra regarding each intra prediction partition. Specifically,first, the intra predicted image generator 12 c identifies a predictionmode based on the intra prediction parameters PP_Intra supplied from thevariable length code decoding unit 11, and assigns the identifiedprediction mode to the target partition, for example, in raster scanorder.

Here, identification of a prediction mode based on the intra predictionparameters PP_Intra can be performed as follows. (1) The estimation flagMPM is decoded, and in the event that this estimation flag MPM indicatesthat the prediction mode regarding the target partition which is aprocessing target agrees with the prediction mode assigned to apartition in the vicinity of this target partition, the prediction modeassigned to a partition in the vicinity of this target partition isassigned to the target partition. (2) On the other hand, in the eventthat this estimation flag MPM indicates that the prediction moderegarding the target partition which is a processing target does notagree with the prediction mode assigned to a partition in the vicinityof this target partition, the residual mode index RIPM is decoded, and aprediction mode that this residual prediction mode index RIPM indicatesis assigned to the target partition.

The intra predicted image generator 12 c generates a predicted imagePred_Intra from the (local) decoded image P by intra-screen predictionin accordance with a prediction procedure that the prediction modeassigned to the target partition indicates. The intra predicted imagePred_Intra generated by the intra predicted image generator 12 c issupplied to the prediction procedure determining unit 12 d. Note that anarrangement may be made wherein the intra predicted image generator 12 cgenerates a predicted image Pred_Intra from the adaptive-filtereddecoded image P_ALF by intra-screen prediction.

(Prediction Procedure Determining Unit 12 d)

The prediction procedure determining unit 12 d determines, based on theprediction type information PT regarding a PU to which each partitionbelongs, whether each partition is an inter prediction partition to besubjected to inter prediction or an intra prediction partition to besubjected to intra prediction. In the case of the former, the predictionprocedure determining unit 12 d supplies the inter predicted imagePred_Inter generated at the inter predicted image generator 12 b to theadder 14 as a predicted image Pred, and in the case of the latter, theprediction procedure determining unit 12 d supplies the intra predictedimage Pred_Intra generated at the intra predicted image generator 12 cto the adder 14 as a predicted image Pred.

(Inverse Quantization/Inverse Transformation Unit 13)

The inverse quantization/inverse transformation unit 13 (1) subjects thequantization prediction residual QD to inverse quantization, (2)subjects a DCT coefficient obtained by inverse quantization to inverseDCT (Discrete Cosine Transform) transform, and (3) supplies a predictionresidual D obtained by inverse DCT transform to the adder 14. Note that,at the time of subjecting the quantization prediction residual QD toinverse quantization, the inverse quantization/inverse transformationunit 13 derives a quantization step QP from the quantization parameterdifference Δqp supplied from the variable length code decoding unit 11.The quantization parameter qp can be derived by adding the quantizationparameter difference Δqp to the quantization parameter qp′ regarding aTU which has been subjected to inverse quantization/inverse DCTtransform immediately before, the quantization step QP can be derivedfrom the quantization step qp by QP=2^(qp/6), for example. Also,generation of the prediction residual D by the inversequantization/inverse transformation unit 13 is performed with a TU or ablock divided from a TU as a unit.

(Adder 14)

The adder 14 generates a decoded image P by adding the predicted imagePred supplied from the predicted image generator 12 and the predictionresidual D supplied from the inverse quantization/inverse transformationunit 13. The generated decoded image P is stored in the frame memory 15.

(Loop Filter 16)

The loop filter 16 has (1) a function as a deblocking filter (DF:Deblocking Filter) configured to perform smoothing (deblocking) of animage in the vicinity of a block boundary or a partition boundary in thedecoded image P, and (2) a function as an adaptive filter (ALF: AdaptiveLoop Filter) configured to perform adaptive filter processing on animage on which the deblocking filter acts, using the filter parametersFP.

FIG. 1 is a block diagram illustrating the configuration of the loopfilter 16. As illustrated in FIG. 1, the loop filter 16 includes adeblocking filter 161, an adaptive filter 162, and internal memory 163.

(Deblocking Filter 161)

The deblocking filter 161 generates a deblocked decoded image P_DB byperforming smoothing of an image in a region in the vicinity of a blockboundary or a partition boundary in the decoded image P.

The deblocking filter 161 performs deblocking on the decoded image P inthe target slice in raster scan order for each LCU as illustrated in (a)in FIG. 4. Also, in the event that the target LCU includes multiple CUs,as illustrated in (b) in FIG. 4, the deblocking filter 161 performsdeblocking in raster scan order for each CU included in the target LCU.Also, regarding the target CU, the deblocking filter 161 first performsdeblocking on a boundary on the left side of the target CU, and thenperforms deblocking on a boundary on the upper side of the target CU.

(c) in FIG. 4 is a diagram illustrating a case where the deblockingfilter 161 acts on the target LCU including four CUs (CU1 to 4). In (c)in FIG. 4, deblocking by the deblocking filter 161 has been applied toboundaries indicated by a thick solid line, and deblocking has not beenapplied to boundaries indicated by a thin solid line even at the time ofdeblocking regarding the target LCU being ended. Note that, of theboundaries indicated by the thin solid lines, deblocking is performed onthe boundary of the right edge of the target LCU at the time ofperforming deblocking on an LCU adjacent to the right side of the targetLCU, and deblocking is performed on the boundary of the lower edge ofthe target LCU at the time of performing deblocking on an LCU adjacentto the lower edge of the target LCU.

In this manner, even at the time of deblocking being performed on thetarget LCU, deblocking has not been performed on the boundaries of theright edge and lower edge of the target LCU, which serves a point forunderstanding at least one aspect of the present invention.

Hereinafter, prior to specific description regarding configuration ofthe units of the loop filter 16, processing by the loop filter 16 willbe schematically described. The loop filter 16 performs one of thefollowing two processing examples (processing example 1, processingexample 2).

Processing Example 1

With the present processing example, the loop filter 16 performsdeblocking in increments of LCUs, and performs adaptive filterprocessing for each first filter processing unit. Note that the firstfilter processing unit in the present processing example is set so as tohave the same size and same position as with the target LCU asillustrated in (a) in FIG. 5. Also, with the following description, thefirst filter processing unit will also be referred to as ALFU1.

(b) in FIG. 5 is a flowchart illustrating a flow of processing in thepresent processing example. The loop filter 16 performs the followingprocessing in the present processing example.

(Step S101)

The loop filter 16 initializes the value of a loop variable n to 1, andstarts first loop processing wherein with regard to n that satisfying1≤n≤N, an increment value of the loop variable n for each loop is takenas 1. Here, N indicates a total number of LCUs included in a slice to beprocessed. Note that, hereinafter, the n'th LCU will also be referred toas LCU(n).

(Step S102)

The loop filter 16 obtains, of the pixel values of the decoded image Pstored in the frame memory 15, a pixel value to be referenced by thedeblocking filter 161 for performing deblocking regarding the LCU(n)from the frame memory 15, and stores this in the internal memory 163.

(Step S103)

Next, the loop filter 16 generates a deblocked image P_DB regarding theLCU(n) by performing deblocking regarding the LCU(n) in the decodedimage P stored in the internal memory 163 using the deblocking filter161, and stores the generated deblocked image P_DB in the internalmemory 163.

(Step S104)

This step is termination of the first loop processing.

(Step S105)

Next, the loop filter 16 initializes the value of a loop variable n to1, and starts second loop processing wherein with regard to n thatsatisfying 1≤n≤N, an increment value of the loop variable n for eachloop is taken as 1. Here, N indicates a total number of ALFU1 s includedin a slice to be processed, and is the same as the total number of LCUs.Note that, hereinafter, the n'th first filter processing unit ALFU1 willalso be referred to as ALFU1(n).

(Step S106)

The loop filter 16 obtains, of the pixel values of the deblocked decodedimage P_DB stored in the frame memory 15, a pixel value to be referencedby the adaptive filter 162 for performing adaptive filter processingregarding the ALFU1(n) from the frame memory 15, and stores this in theinternal memory 163.

(Step S107)

Next, the loop filter 16 generates a filtered image P_ALF regarding theALFU1(n) by performing adaptive filter processing regarding the ALFU1(n)in the deblocked decoded image P_DB stored in the internal memory 163using the adaptive filter 162, and stores the generated filtered imageP_ALF in the internal memory 163.

(Step S108)

This step is termination of the second loop.

The above is the flow of the processing in the processing example 1.Note that the filtered image P_ALF stored in the internal memory 163 bythe adaptive filter 162 is transferred to and stored in the frame memory15.

In this manner, with the present processing example, deblocking isperformed previously on all of the LCUs included in the target slice,and adaptive filter processing is then performed. Accordingly, with thepresent processing example, the loop filter 16 performs transfer of datafrom the frame memory 15 to the internal memory 163 twice regarding eachn (step S102 and step S106).

Processing Example 2

With the present processing example, the loop filter 16 performsdeblocking in increments of LCUs, and performs adaptive filterprocessing for each second filter processing unit. Note that the secondfilter processing unit in the present processing example is set so as tohave the same size as with the target LCU and so as to have a positiondifferent from the target LCU as illustrated in (c) in FIG. 5. Also, thesecond filter processing unit regarding the target LCU is set so as notto be overlapped with a region where deblocking has not been performedregarding the target LCU (shaded portion in (c) in FIG. 5). Also, withthe following description, the second filter processing unit will alsobe referred to as ALFU2.

(d) in FIG. 5 is a flowchart illustrating a flow of processing in thepresent processing example. The loop filter 16 performs the followingprocessing in the present processing example.

(Step S201)

The loop filter 16 initializes the value of a loop variable n to 1, andstarts loop processing wherein with regard to n that satisfying 1≤n≤N,an increment value of the loop variable n for each loop is taken as 1.Here, as described above, N indicates a total number of LCUs included ina slice to be processed. Note that, hereinafter, the n'th second filterprocessing unit ALFU2 will also be referred to as ALFU2(n).

(Step S202)

The loop filter 16 obtains, of the pixel values of the decoded image Pstored in the frame memory 15, a pixel value to be referenced forperforming deblocking regarding the LCU(n) and adaptive filterprocessing regarding the ALFU2(n) from the frame memory 15, and storesthis in the internal memory 163.

(Step S203)

Next, the loop filter 16 generates a deblocked image P_DB regarding theLCU(n) by performing deblocking regarding the LCU(n) in the decodedimage P stored in the internal memory 163 using the deblocking filter161, and stores the generated deblocked image P_DB in the internalmemory 163.

(Step S204)

Next, the loop filter 16 generates a filtered image P_ALF regarding theALFU2(n) by performing adaptive filter processing regarding the ALFU2(n)in the deblocked decoded image P_DB stored in the internal memory 163using the adaptive filter 162, and stores the generated filtered imageP_ALF in the internal memory 163.

(Step S205)

This step is termination of the loop.

The above is the flow of the processing in the processing example 2.Note that the filtered image P_ALF stored in the internal memory 163 bythe adaptive filter 162 is transferred to and stored in the frame memory15.

In this manner, with the present processing example, after deblocking isperformed on the LCU(n) included in the target slice, and adaptivefilter processing in the ALFU2(n) corresponding to the LCU(n) isperformed. Accordingly, with the present processing example, it issufficient that the loop filter 16 performs transfer of data from theframe memory 15 to the internal memory 163 once regarding each n (stepS202).

(Adaptive Filter 162)

Hereinafter, the configuration of the adaptive filter 162 included inthe loop filter 16 will be described with reference to FIG. 1. Asillustrated in FIG. 1, the adaptive filter 162 includes a target pixelsetting unit 162 a, a reference region setting unit 162 b, and a filterprocessing unit 162 c.

The target pixel setting unit 162 a sets the pixels included in thefilter processing unit ALFU to as the target pixel in raster scan order.Target pixel position information that indicates the position of thetarget pixel set by the target pixel setting unit 162 a is supplied tothe reference region setting unit 162 b. The reference region R is setfor each target pixel by the reference region setting unit 162 b, and apixel value is calculated by the filter processing unit 162 c for eachtarget pixel.

The reference region setting unit 162 b sets a filter reference region R(also simply referred to as reference region R) in the vicinity of thetarget pixel based on the position of the target pixel within the filterprocessing unit ALFU or the position of the target pixel within thetarget LCU. Here, the filter reference region R is determined dependingon at least one of (1) tap information included in the filter parametersFP, (2) width Ddf in the Y direction of a region where deblocking by thedeblocking filter 161 has not been completed in the target LCU, (3)length LLCUy in the Y direction of the target LCU, and (4) an absolutevalue Ly of deviation in the Y direction between the target LCU andtarget ALFU. Note that the length LLCUy in the Y direction of the targetLCU may be expressed as height LcuHeight of the LCU. Note that the LLCUy(LcuHeight) is derived with the following expression using a logarithmicvalue Log 2MaxCUSize of the LCU size.

LcuHeight=(1<<Log 2MaxCUSize)

The reference region setting unit 162 b sets the reference region R soas to satisfy at least one restraint of restraints A to E listed below.

Restraint A: In the case that the Y coordinate within the LCU of thetarget pixel is less than a predetermined threshold Lupper1, an upwardreference range Dupper is restricted to the number of pixels equal to orsmaller than the Y coordinate within the LCU of the target pixel.

The upward reference range Dupper mentioned here specifies distancealong the Y axis between the target pixel and a pixel within thereference region R which is a pixel adjacent to the upper edge of thereference region R (this will be applied to the following). Also, let ussay that the upward reference range Dupper is represented with a pixelas a unit (this will be applied to the following).

Also, as the predetermined threshold Lupper1, a fixed value such as 4 or8 or the like may be employed, or a variable value may be employed (thiswill be applied to the following). Also, the upward reference rangeDupper in the event that the Y coordinate within the LCU of the targetpixel is less than the predetermined threshold Lupper1 may be set so asto depend on the Y coordinate within the LCU of the target pixel, or maybe set to, for example, 0 without depending on the Y coordinate withinthe LCU of the target pixel.

In more general, the restraint A may also be expressed as a restraintfor restricting, when distance DU between the upstream edge of the LCUincluding the target pixel, and this target pixel is less than apredetermined threshold, a position of the upstream edge of thereference region R regarding this target pixel so that distance withthis target pixel is equal to or shorter than the DU.

Note that the above-mentioned “upstream edge” specifies any one of theupper edge and left edge when the processing is performed in raster scanorder, and specifies any one of the lower edge and right edge when theprocessing is performed in inverse order of raster scan. Also, theabove-mentioned “downstream edge” specifies any one of the lower edgeand right edge when the processing is performed in raster scan order,and specifies any one of the upper edge and left edge when theprocessing is performed in inverse order of raster scan (this will beapplied to the following).

Also, with the following description, “the downstream side of thedownstream edge” specifies, when the processing is performed in rasterscan order, any one of “the lower side of the lower edge”, and “theright side of the right edge”, and specifies, when the processing isperformed in inverse order of raster scan, any one of “the upper side ofthe upper edge”, and “the left side of the left edge”. Also, with thefollowing description, “the upstream side of the downstream edge”specifies, when the processing is performed in raster scan order, anyone of “the upper side of the lower edge”, and “the left side of theright edge”, and specifies, when the processing is performed in inverseorder of raster scan, any one of “the lower side of the upper edge”, and“the right side of the left edge”.

Restraint B: In the case that the Y coordinate within the LCU of thetarget pixel is equal to or greater than a predetermined thresholdLLCUy−Llower1, a downward reference range Dlower is restricted to thenumber of pixels equal to or smaller than LLCUy−(Y coordinate within theLCU of the target pixel)−1.

The downward reference range Dlower mentioned here specifies distancealong the Y axis between the target pixel and a pixel within thereference region R which is a pixel adjacent to the lower edge of thereference region R (this will be applied to the following). Also, let ussay that the downward reference range Dlower is represented with a pixelas a unit (this will be applied to the following).

Also, the downward reference range Dlower in the event that the Ycoordinate within the LCU of the target pixel is equal to or greaterthan the predetermined threshold LLCUy−Llower1 may be set so as todepend on the Y coordinate within the LCU of the target pixel, or may beset to, for example, 0 without depending on the Y coordinate within theLCU of the target pixel.

In more general, the restraint B may also be expressed as a restraintfor restricting, when distance DL between the downstream edge of the LCUincluding the target pixel, and this target pixel is less than apredetermined threshold, a position of the downstream edge of thereference region regarding this target pixel so that distance with thistarget pixel is equal to or shorter than the DL.

Restraint C: In the case that the normalized Y coordinate within the LCUof the target pixel is equal to or greater than a predeterminedthreshold Lupper2, the upward reference range Dupper is restricted tothe number of pixels equal to or smaller than Ly+(normalized Ycoordinate within the LCU of the target pixel)−LLCUy.

A boundary of the Lupper2 indicated with the predetermined thresholdwill be called virtual boundary line (Virtual Boundary Line). Theposition of the virtual boundary line will be referred to as VBLine.

The normalized Y coordinate within the LCU mentioned here is definedwith (Y coordinate within the LCU) % LLCUy in which a symbol “%” thatrepresents remainder operation is used, and takes a value equal to orgreater than 0 (this will be applied to the following). Also, the Ycoordinate within the LCU=(Y coordinate of the target pixel) % LLCUy,and consequently, the normalized Y coordinate within the LCU is the sameas (Y coordinate % LLCUy) using the Y coordinate of the target pixel.Also, in the event that the position of the target pixel is expressedwith the upper left coordinates (xC, yC) of a block including the targetpixel and relative coordinates (x, y) from the upper left coordinates ofthe block, normalized Y coordinate yr within the LCU is represented with(yC+y) % LcuHeight.

Also, the predetermined threshold Lupper2 has to be a value equal to orsmaller than LLCUy−Ly (this will be applied to the following). If we saythat Ly=4, the position of the virtual boundary line in the restraint CVBLine (=LcuHeight−Ly) becomes LcuHeight−4. If we say that Ly=3, theVBLine becomes LcuHeight−3. Also, the upward reference range Dupper inthe event that the normalized Y coordinate within the LCU of the targetpixel is equal to or greater than the predetermined threshold Lupper2may be set so as to depend on the Y coordinate within the LCU of thetarget pixel, or may be set to, for example, 0 without depending on theY coordinate within the LCU of the target pixel.

In more general, the restraint C may also be expressed as a restraintfor restricting, when the target pixel is positioned on the downstreamside of the downstream edge of the ALFU corresponding to the CLUincluding this target pixel (equal to or lower than the virtual boundaryline VBLine), the position of the upstream edge of the reference regionR regarding the target pixel so that distance with this target pixelbecomes equal to or shorter than distance between this target pixel andthe upstream edge of the output unit thereof (virtual boundary lineVBLine). In this case, the reference region R is restricted so as not toinclude a pixel on the upper side than the virtual boundary line VBLine.That is to say, a pixel alone of which the Y coordinate is equal to orlower than the virtual boundary line VBLine is referenced.

Restraint D: In the case that the normalized Y coordinate within the LCUof the target pixel is less than a predetermined threshold Llower2, thedownward reference range Dlower is restricted to the number of pixelsequal to or smaller than LLCUy−Ddf−(normalized Y coordinate within theLCU of the target pixel)−1.

Here, the downward reference range Dlower in the event that thenormalized Y coordinate within the LCU of the target pixel is less thanthe predetermined threshold Llower2 may be set so as to depend on the Ycoordinate within the LCU of the target pixel, or may be set to, forexample, 0 without depending on the Y coordinate within the LCU of thetarget pixel. Also, the threshold Llower2 has to be a value equal to orgreater than LLCU y−Ly.

In more general, the restraint D may also be expressed as a restraintfor restricting, when the target pixel is positioned on the upstreamside of the downstream edge of the ALFU corresponding to the CLUincluding this target pixel, the position of the downstream edge of thereference region R regarding the target pixel so that distance with thistarget pixel becomes equal to or shorter than distance to be obtained bysubtracting Ddf from distance between this target pixel and thedownstream edge of this LCU.

Restraint E: In the case that the normalized Y coordinate within the LCUof the target pixel is less than a predetermined threshold Llower3, thedownward reference range Dlower is restricted to the number of pixelsequal to or smaller than LLCUy−Ly−(normalized Y coordinate within theLCU of the target pixel)−1. A boundary of Lupper3 indicated with apredetermined threshold will also be called a virtual boundary line(Virtual Boundary Line).

Here, the threshold Llower3 has to be a value equal to or greater thanLLCUy−Ly. Now, if we say that Ly=4, a position of the virtual boundaryline VBLine (=LcuHeight−Ly) in the restraint E becomes LcuHeight−4. Ifwe say that Ly=3, the VBLine becomes LcuHeight−3.

In more general, the restraint E may also be expressed as a restraintfor restricting, when the target pixel is positioned on the upstreamside of the downstream edge of the ALFU corresponding to the CLUincluding this target pixel (upper side than the virtual boundary lineVBLine), the position of the downstream edge of the reference region Rregarding this target pixel so that distance with this target pixelbecomes equal to or shorter than distance between this target pixel andthe downstream edge of the output unit thereof (virtual boundary lineVBLine). In this case, the reference region R is restricted so as not toinclude a pixel below the virtual boundary line VBLine. That is to say,a pixel alone of which the Y coordinate is above the virtual boundaryline VBLine is referenced.

(Filter Processing Unit 162 c)

The filter processing unit 162 c calculates the pixel value of thetarget pixel of the adaptive-filtered decoded image P_ALF (also referredto as “image after filtering”) with reference to the pixel value of apixel included in the reference region R set regarding this target pixelby the reference region setting unit 162 b which is a pixel value of thedeblocked decoded image P_DB (also referred to as “image beforefiltering”). The generated adaptive-filtered decoded image P_ALF isstored once in the internal memory 163, and then transferred to theframe memory 15.

If we represent the pixel value of the target pixel in theadaptive-filtered decoded image P_ALF as SF(x′, y′), and represent apixel value of the deblocked decoded image P_DB in the filter referencerange as S(x, y), the filter processing unit 162 c calculates the pixelvalue SF(x′, y′) using the following Expression (1).

$\begin{matrix}\left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 1} \right\rbrack & \; \\{{S_{F}\left( {x^{\prime},y^{\prime}} \right)} = {\left( {\sum\limits_{{({i,j})} \in R}{{a\left( {i,j} \right)} \times {S\left( {{x + i},{y + j}} \right)}}} \right) + o}} & (1)\end{matrix}$

Now, the coordinates (x, y) may be the same as the coordinates (x′, y′),or may be different coordinates as long as these have correspondence ofone on one. Also, a(i, j) represents of filter coefficients included inthe filter parameters FP, a filter coefficient to be multiplied on apixel value S(x+i, y+j) of an image before filtering, and o representsoffset included in the filter parameters FP.

Also, R represents a region to be referenced for calculating the pixelvalue of the target pixel, and is set by the reference region settingunit 162 b. (a) in FIG. 6 illustrates an allocation example of filtercoefficients to the pixels included in the reference region R of 5×5taps set by the reference region setting unit 162 b. In (a) in FIG. 6, ashaded pixel indicates the target pixel S(x′, y′). As illustrated in (a)in FIG. 6, each filter coefficient can be configured so as to beallocated to each pixel included in the filter reference region R so asto have a rotational symmetry property of 180 degrees. Specifically,aI(i, j) in Expression (1) can be configured such as aI (−3, −3)=aI(3,3)=a0, aI (−2, −3)=aI(2, 3)=a1, . . . , using the filter coefficients a0to a12 included in the filter parameters FP so as to satisfy aI(i,j)=aI(−i, −j). (b) in FIG. 6 illustrates an allocation example of thefilter coefficients to the pixels included in the reference region R of5 (horizontal direction)×3 (vertical direction) taps set by thereference region setting unit 162 b.

(Flow of Adaptive Filter Processing)

Hereinafter, generation processing of the adaptive-filtered decodedimage P_ALF will be described with reference to FIG. 7 to FIG. 9. FIG. 7is a flowchart illustrating a flow of the generation processing of theadaptive-filtered decoded image P_ALF by the adaptive filter 162.

(Step S1)

The adaptive filter 162 initializes any of values of loop variables xxand yy to 0 using the target pixel setting unit 162 a, and starts firstloop processing wherein an increment value of the loop variable xx whichsatisfies 0≤xx≤LLCUx−1 is set to 1, and second loop processing whereinan increment value of the loop variable yy which satisfies 0≤yy≤LLCUy−1is set to 1. Here, (xx, yy) represents coordinates within the ALFU ofthe target pixel. Also, the LLCUx and LLCUy represent the length in thehorizontal direction of the target LCU and the length in the verticaldirection of the target LCU, respectively.

(Step S2)

Next, the adaptive filter 162 set the upward reference range Dupper anddownward reference range Dlower to

Dupper=Dtap

Dlower=Dtap

with reference to tap count information included in the filterparameters FP using the reference region setting unit 162 b. Now, aspecific value of the Dtap is determined according to the number of tapsthat the tap count information indicates. For example, when the numberof taps that the tap count information indicates is five taps, Dtap=2,when the number of taps that the tap count information indicates isseven taps, Dtap=3, and when the number of taps that the tap countinformation indicates is nine taps, Dtap=4. In general, when the numberof taps that the tap count information indicates is N taps,Dtap=(Ntap−1)/2.

(Step S3)

Next, the adaptive filter 162 converts the coordinates of the targetpixel from the coordinates within the ALFU to the normalized coordinateswithin the LCU at the reference region setting unit 162 b. Specifically,the normalized coordinates (xr, yr) within the LCU of the target pixelis calculated from the coordinates (xx, yy) within the ALFU of thistarget pixel as follows.

yr=(yy−Ly)% LLCUy

xr=(xx−Lx)% LLCUx

(Step S4)

Next, the adaptive filter 162 sets the reference region R according tothe values of (xr, yr) using the reference region setting unit 162 b.

(Step S5)

Next, the adaptive filter 162 calculates the pixel value of the targetpixel with reference to each pixel value of the deblocked decoded imageP_DB in the reference region R set in step S1004 using the filterprocessing unit 162 c.

(Step S6)

This step is termination of the first loop and second loop.

The above is the flow of the generation processing of theadaptive-filtered decoded image P_ALF in the target LCU. Here, thereference region R in step S4 is set so as to satisfy at least onerestraint of the restraints A to E. Hereinafter, a setting example ofthe reference region R in step S4 will be described with reference toFIG. 8 and FIG. 9.

Setting Example 1

The reference region setting unit 162 b sets the reference region R soas to satisfy the restraint A. Specifically, the reference regionsetting unit 162 b performs, in the above-mentioned step S4, processingillustrated in (a) in FIG. 8, that is, discriminant processing regardingwhether or not the normalized Y coordinate yr within the LCU is lessthan a predetermined threshold Lupper1 (step S4 a 1), and in the eventthat the normalized Y coordinate yr within the LCU is less than thepredetermined threshold Lupper1, performs processing for setting thereference region R so that the upward reference range Dupper satisfiesDupper≤yr (step S4 a 2).

The reference region setting unit 162 b performs the processing of thepresent example, and accordingly, the filter processing unit 162 cgenerates the adaptive-filtered decoded image P_ALF without referencingthe pixel value on the upper side of the filter processing unit ALFU.Accordingly, the amount of data to be transferred from the frame memory15 to the internal memory 163, and the amount of data to be transferredfrom the internal memory 163 to the filter processing unit 162 c arereduced.

Note that the processing of the present example may suitably be appliedeven when the loop filter 16 performs any of the above-mentionedprocessing example 1 and processing example 2.

(a) in FIG. 9 is a diagram illustrating an example of the referenceregion R set by the reference region setting unit 162 b which performsthe processing of the present example, with regard to the target LCU.(a) in FIG. 9 illustrates a case where when the normalized Y coordinateyr within the LCU of the target pixel is less than the thresholdLupper1, the upward reference range Dupper is set to 0.

Setting Example 2

The reference region setting unit 162 b sets the reference region R soas to satisfy the constraint B. Specifically, the reference regionsetting unit 162 b performs, in the above-mentioned step S4, processingillustrated in (b) in FIG. 8, that is, discriminant processing regardingwhether or not the normalized Y coordinate yr within the LCU is equal toor greater than a predetermined threshold LLCUy−Llower1 (step S4 b 1),and in the event that the normalized Y coordinate yr within the LCU isequal to or greater than a predetermined threshold LLCUy−Llower1,performs processing for setting the reference region R so that thedownward reference range Dlower satisfies Dlower≤LLCUy−yr−1 (step S4 b2).

The reference region setting unit 162 b performs the processing of thepresent example, and accordingly, the filter processing unit 162 cgenerates the adaptive-filtered decoded image P_ALF without referencingthe pixel value on the lower side of the filter processing unit ALFU.Accordingly, the amount of data to be transferred from the frame memory15 to the internal memory 163, and the amount of data to be transferredfrom the internal memory 163 to the filter processing unit 162 c arereduced. Also, the filter processing unit 162 c can perform filterprocessing without awaiting until decoding of the LCU on the lower sideof the target LCU is completed, and accordingly, processing time isreduced.

Note that the processing of the present example may suitably be appliedeven when the loop filter 16 performs any of the above-mentionedprocessing example 1 and processing example 2.

(b) in FIG. 9 is a diagram illustrating an example of the referenceregion R set by the reference region setting unit 162 b which performsthe processing of the present example, with regard to the target LCU.(b) in FIG. 9 illustrates a case where when the normalized Y coordinateyr within the LCU of the target pixel is equal to or greater than thethreshold LLCUy−Llower1, the downward reference range Dlower is set to0.

Setting Example 3

The reference region setting unit 162 b sets the reference region R soas to satisfy both of the constraint A and constraint B. Specifically,the reference region setting unit 162 b performs, in the above-mentionedstep S4, both of the processing illustrated in (a) in FIG. 8 (step S4 a1, step S4 a 2) and the processing illustrated in (b) in FIG. 8 (step S4b 1, step S4 b 2).

The reference region setting unit 162 b performs the processing of thepresent example, and accordingly, the filter processing unit 162 cgenerates the adaptive-filtered decoded image P_ALF without referencingthe pixel value on the upper side and the pixel value on the lower sideof the filter processing unit ALFU. Accordingly, the amount of data tobe transferred from the frame memory 15 to the internal memory 163, andthe amount of data to be transferred from the internal memory 163 to thefilter processing unit 162 c are reduced. Also, the filter processingunit 162 c can perform filter processing without awaiting until decodingof the LCU on the lower side of the target LCU is completed, andaccordingly, processing time is reduced.

Note that the processing of the present example may suitably be appliedeven when the loop filter 16 performs any of the above-mentionedprocessing example 1 and processing example 2.

(c) in FIG. 9 is a diagram illustrating an example of the referenceregion R set by the reference region setting unit 162 b which performsthe processing of the present example, with regard to the target LCU.(c) in FIG. 9 illustrates a case where when the normalized Y coordinateyr within the LCU of the target pixel is less than a threshold Lupper1,the upward reference range Dupper is set to 0, and when the normalized Ycoordinate yr within the LCU of the target pixel is equal to or greaterthan a threshold LLCUy−Llower1, the downward reference range Dlower isset to 0.

Setting Example 4

The reference region setting unit 162 b sets the reference region R soas to satisfy the constraint D. Specifically, the reference regionsetting unit 162 b performs, in the above-mentioned step S4, processingillustrated in (d) in FIG. 8, that is, discriminant processing regardingwhether or not the normalized Y coordinate yr within the LCU is lessthan a predetermined threshold Llower2 (step S4 d 1), and in the eventthat the normalized Y coordinate yr within the LCU is less than thepredetermined threshold Llower2, performs processing for setting thereference region R so that the downward reference range Dlower satisfiesDlower≤LLCUy−Ddf−yr−1 (step S4 d 2). Here, it is desirable that thethreshold Llower2 satisfies Llower2=LLCUy−Ly (this will be applied tothe following).

Note that, with the present example, the reference region setting unit162 b may be of a configuration so as to discriminate, in step S4 d 1,whether or not the normalized Y coordinate yr within the LCU is equal toor greater than a predetermined threshold Llower2′, and also less than apredetermined threshold Llower2, and in the event that the normalized Ycoordinate yr within the LCU is equal to or greater than a predeterminedthreshold Llower2′, and also less than a predetermined thresholdLlower2, in step S4 d 2 set the reference region R so that the downwardreference range Dlower satisfies Dlower≤LLCUy−Ddf−yr−1. Here, it isdesirable that the threshold Llower2′ satisfies Llower2′=LLCUy−Ly−Dtap.

The processing of the present example can suitably be applied in theevent that the loop filter 16 performs the processing in theabove-mentioned processing example 2. Accordingly, the reference regionsetting unit 162 b performs the processing of the present example, andaccordingly, it is sufficient that the loop filter 16 performs transferof data from the frame memory 15 to the internal memory 163 onceregarding each LCU (step S202), and accordingly, the amount of data tobe transferred from the frame memory 15 to the internal memory 163 canbe reduced. Also, the filter processing unit 162 c can perform filterprocessing without awaiting until deblocking regarding the lower edge ofthe target LCU is completed, and accordingly, the processing time isreduced.

(d) in FIG. 9 is a diagram illustrating an example of the referenceregion R set by the reference region setting unit 162 b which performsthe processing of the present example, with regard to the target LCU.(d) in FIG. 9 illustrates a case where when the normalized Y coordinateyr within the LCU of the target pixel is equal to or greater than athreshold Llower2′, and also less than the threshold Llower2, thedownward reference range Dlower is set to 0.

Setting Example 5

The reference region setting unit 162 b sets the reference region R soas to satisfy both of the constraint C and constraint D. Specifically,the reference region setting unit 162 b performs, in the above-mentionedstep S4, the processing illustrated in (c) in FIG. 8, that is,discriminant processing regarding whether or not the normalized Ycoordinate yr within the LCU is equal to or greater than a predeterminedthreshold Lupper2 (equal to or lower than the virtual boundary lineVBLine) (step S4 c 1), and in the event that the normalized Y coordinateyr within the LCU is equal to or greater than a predetermined thresholdLupper2 (in the event of being positioned equal to or lower than thevirtual boundary line VBLine), performs processing for setting thereference region R so that the upward reference range Dupper satisfiesDupper≤Ly+yr−LLCUy (step S4 c 2). In this case, a pixel alone of the Ycoordinate is equal to or lower than the virtual boundary line VBLine isreferenced. Here, it is desirable that the threshold Lupper2 satisfiesLupper2=LLCUy−Ly. Also, the reference region setting 162 b also performsthe processing illustrated in (d) in FIG. 8 (step S4 d 1, step S4 d 2).The above-mentioned discriminant processing may be expressed asdist2VB≥0 by using difference between the virtual boundary line VBLineand the normalized Y coordinate yr within the LCU dist2VB=(y coordinate% LcuHeight)−VBLine.

Note that, with the present example, the reference region setting unit162 b may be configured so as to discriminate, in step S4 c 1, whetheror not the normalized Y coordinate yr within the LCU is equal to orgreater than a predetermined threshold Lupper2, and also less than apredetermined threshold Lupper2′, and in the event that the normalized Ycoordinate yr within the LCU is equal to or greater than thepredetermined threshold Lupper2, and also less than a predeterminedthreshold Lupper2′, in step S4 c 2 set the reference region R so thatthe upward reference range Dupper satisfies Dupper≤Ly+yr−LLCUy. Here, itis desirable that the threshold Lupper2′ satisfiesLupper2′=LLCUy−Ly+Dtap.

The processing of the present example can suitably be applied in theevent that the loop filter 16 performs the processing in theabove-mentioned processing example 2. Accordingly, the reference regionsetting unit 162 b performs the processing of the present example, andaccordingly, it is sufficient that the loop filter 16 performs transferof data from the frame memory 15 to the internal memory 163 onceregarding each LCU (step S202), and accordingly, the amount of data tobe transferred from the frame memory 15 to the internal memory 163 canbe reduced. Also, the filter processing unit 162 c generates theadaptive-filtered decoded image P_ALF without referencing a pixel valueon the upper side of the filter processing unit ALFU. Accordingly, theamount of data to be transferred from the frame memory 15 to theinternal memory 163, and the amount of data to be transferred from theinternal memory 163 to the filter processing unit 162 c are reduced.Also, the filter processing unit 162 c can perform filter processingwithout awaiting until deblocking regarding the lower edge of the targetLCU is completed, and accordingly, processing time is reduced.

(e) in FIG. 9 is a diagram illustrating an example of the referenceregion R set by the reference region setting unit 162 b which performsthe processing of the present example, with regard to the target LCU.(e) in FIG. 9 illustrates a case where Lupper2=Llower2′, but the presentembodiment is not restricted to this.

Setting Example 6

The reference region setting unit 162 b sets the reference region R soas to satisfy all of the constraint A, constraint B, constraint C, andconstraint D. Specifically, the reference region setting unit 162 bperforms all of the processing illustrated in (a) to (d) in FIG. 8.

Note that the processing of the present example may suitably be appliedeven when the loop filter 16 performs any of the above-mentionedprocessing example 1 and processing example 2.

In the event that the loop filter 16 performs the processing in theabove-mentioned processing example 2, the reference region setting unit162 b performs the processing of the present example, and accordingly,it is sufficient that the loop filter 16 performs transfer of data fromthe frame memory 15 to the internal memory 163 once regarding each LCU(step S202), and accordingly, the amount of data to be transferred fromthe frame memory 15 to the internal memory 163 can be reduced. Also, thefilter processing unit 162 c can perform filter processing withoutawaiting until deblocking regarding the lower edge of the target LCU iscompleted, and accordingly, the processing time is reduced.

Also, even when the loop filter 16 performs any processing of theabove-mentioned processing example 1 and processing example 2, thereference region setting unit 162 b performs the processing of thepresent example, and accordingly, the filter processing unit 162 cgenerates the adaptive-filtered decoded image P_ALF without referencinga pixel value on the upper side of the filter processing unit ALFU.Accordingly, the amount of data to be transferred from the frame memory15 to the internal memory 163, and the amount of data to be transferredfrom the internal memory 163 to the filter processing unit 162 c arereduced.

(f) in FIG. 9 is a diagram illustrating an example of the referenceregion R set by the reference region setting unit 162 b which performsthe processing of the present example, with regard to the target LCU.(f) in FIG. 9 illustrates a case where Lupper2=Llower2′, but the presentembodiment is not restricted to this.

Setting Example 7

The reference region setting unit 162 b sets the reference region R soas to satisfy the constraint E. Specifically, the reference regionsetting unit 162 b performs, in the above-mentioned step S4, processingillustrated in (e) in FIG. 8, that is, discriminant processing regardingwhether or not the normalized Y coordinate yr within the LCU is lessthan a predetermined threshold Lupper3 (upper than the virtual boundaryline VBLine) (step S4 e 1), and in the event that the normalized Ycoordinate yr within the LCU is less than the predetermined thresholdLupper3 (in the event of being positioned upper than the virtualboundary line VBLine), performs processing for setting the referenceregion R so that the downward reference range Dlower satisfiesDlower≤LLCUy−Ly−yr−1 (step S4 e 2). In this case, a pixel alone of whichthe Y coordinate is upper than the virtual boundary line VBLine isreferenced. Here, it is desirable that the threshold Lupper3 satisfiesLupper3=LLCUy−Ly. The above-mentioned discriminant processing may beexpressed as dist2VB<0 by using difference between the virtual boundaryline VBLine and the normalized Y coordinate yr within the LCU dist2VB=(ycoordinate % LcuHeight)−VBLine.

(More Detailed Description Regarding Constraint Imposed on ReferenceRegion)

Hereinafter, the above-mentioned restraints A to E, and otherexpressions other than the constraints A to E will be described indetail step by step.

(Coordinate System)

First, a coordinate system to be used for expressing the constraints Ato E will be described. Note that, with the following description, thefirst filter processing unit ALFU1 and second filter processing unitALFU2 are collectively named as “filter processing unit ALFU”.

FIG. 10 is a diagram illustrating a position relation between the targetLCU and the filter processing unit ALFU corresponding to the target LCU.As illustrated in FIG. 10, the coordinates of each pixel in the LCU areexpressed as (x, y) using an X coordinate x and a Y coordinate y withinthe LCU with an upper left pixel within the LCU as the origin. Also, thecoordinates of each pixel in the ALFU are expressed as (xx, yy) using anX coordinate xx and a Y coordinate yy within the ALFU with an upper leftpixel within the ALFU as the origin. Note that the coordinates (x, y)within the LCU and coordinates (xx, yy) within the ALFU are expressedwith a pixel as a unit and with the downstream direction of raster scanas a positive.

As illustrated in FIG. 10, the ALFU is set to a location shifted by Lyon the upstream side of raster scan along the Y axis and by Lx on theupstream side of raster scan along the X axis as compared to the LCU.Accordingly, a relation between the (x, y) and (xx, yy) is expressed asfollows.

x=xx−Lx

y=yy−Ly

Also, the coordinates of each pixel in the LCU may also be expressed bynormalized coordinates (xr, yr) within the LCU defined as follows.

xr=x % LLCUx

yr=y % LLCUy

Here, the symbol % represents remainder operation, LLCUx representslength in the x direction of the target LCU, and LLCUy represents lengthin the y direction of the target LCU. Also, in the event that the loopfilter 16 performs the processing of the above-mentioned processingexample 1, that is, in the event that the ALFU is ALFU2, Lx=Ly=0 issatisfied.

(a) in FIG. 11 is a diagram illustrating a position in the ALFU of thetarget pixel of which the pixel value is calculated by the adaptivefilter 162, and (b) in FIG. 11 is a diagram illustrating the referenceregion R which is a region made up of a pixel to be referenced by theadaptive filter 162 for calculating the pixel value of the target pixel.

As illustrated in (b) in FIG. 11, the reference region R is set so thatdistance along the Y axis between the target pixel and a pixel withinthe reference region R which is a pixel adjacent to the upper edge ofthe reference region R makes up the upward reference range DDupper, anddistance along the Y axis between the target pixel and a pixel withinthe reference region R which is a pixel adjacent to the lower edge ofthe reference region R makes up the downward reference range DDlower.Here, specific values of the upward reference range DDupper and downwardreference range DDlower are set by the adaptive filter 162 according tothe tap count information included in the filter parameters FP and theposition of the target pixel. For example, in the event that the tapcount information indicates nine taps, a value that the upward referencerange DDupper can take is any of 0, 1, 3, and 4. This is also the sameas with the downward reference range DDlower.

Note that the upward reference range and downward reference range at thetime of coordinates within the ALFU of the target pixel being (xx, yy)are represented as DDupper(yy) and DDlower(yy). Also, the upwardreference range and downward reference range at the time of coordinateswithin the LCU of the target pixel being (x, y) are represented asDupper(y) and Dlower(y).

(Constraints to be Imposed on Reference Region R by Adaptive Filter 161)

Next, description will be made regarding constraints to be imposed onthe upward reference range DDupper and downward reference range DDlowerwhen the adaptive filter 162 sets the reference region R regarding thetarget pixel. Note that the above-mentioned constraints A to E arerestraints which express an upper restraint, a first lower restraint,and a second lower restraint, which will be described below, usingnormalized coordinates within the LCU. Accordingly, the adaptive filter162 may also be expressed as a filter configured to set the referenceregion R by imposing at least one restraint of the upper restraint,first lower restraint, and second lower restraint which will bedescribed below.

(Upper Restraint and First Lower Restraint)

The adaptive filter 162 sets, in the event that the loop filter 161performs one of the above-mentioned processing example 1 and processingexample 2, the reference region R so as to satisfy at least one of theupper restraint and first lower restraint which will be described below.

Upper restraint: The pixel value of a pixel positioned above the ALFU1is not referenced at the time of calculating the pixel value of thetarget pixel.

First lower restraint: The pixel value of a pixel positioned below theALFU1 is not referenced at the time of calculating the pixel value ofthe target pixel.

The adaptive filter 162 sets the reference region R so as to satisfy atleast one of the above-mentioned two restraints, whereby the amount ofdata to be transferred in the above-mentioned step S106 can be reduced.

FIG. 12 is a diagram for describing a reduction effect of the amount ofdata to be transferred in the case that the adaptive filter 162 whichperforms the processing of the processing example 1 imposes at least oneof the above-mentioned upper restraint and first lower restraint on thereference region R.

(a) in FIG. 12 is a diagram illustrating regions of pixels to betransferred in the adaptive filter processing according to the relatedart where neither the above-mentioned upper restraint nor theabove-mentioned first lower restraint are imposed. As illustrated in (a)in FIG. 12, with the adaptive filter processing according to the relatedart, there have to be transferred the pixel value of a pixel positionedabove the target LCU and the pixel value of a pixel positioned below thetarget LCU which are pixels in the vicinity of the target LCU.

(b) in FIG. 12 is a diagram illustrating regions of pixels to betransferred in the adaptive filter processing in the event that theadaptive filter 162 imposes the first lower restraint. As illustrated in(b) in FIG. 12, in the event of imposing the first lower restraint, theadaptive filter processing can be performed without transferring thepixel values of pixels positioned below the target LCU.

(c) in FIG. 12 is a diagram illustrating regions of pixels to betransferred in the adaptive filter processing in the event that theadaptive filter 162 imposes the upper restraint. As illustrated in (c)in FIG. 12, in the event of imposing the upper restraint, the adaptivefilter processing can be performed without transferring the pixel valuesof pixels positioned above the target LCU.

(d) in FIG. 12 is a diagram illustrating regions of pixels to betransferred in the adaptive filter processing in the event that theadaptive filter 162 imposes both of the upper restraint and first lowerrestraint. As illustrated in (d) in FIG. 12, in the event of imposingboth of the upper restraint and first lower restraint, the adaptivefilter processing can be performed without transferring the pixel valuesof pixels positioned above and below the target LCU.

(Second Lower Restraint)

The adaptive filter 162 sets, in the event that the loop filter 161performs the processing illustrated in the above-mentioned processingexample 2, the reference region R so as to satisfy a later-describedsecond lower constraint.

Second lower restraint: A region where deblocking in the target LCU hasnot been completed is not referenced at the time of calculating thepixel value of the target pixel.

The adaptive filter 162 can perform the processing of the processingexample 2 without awaiting until deblocking is completed regarding aregion where deblocking in the target LCU has not been completed bysetting the reference region R so as to satisfy the above-mentionedsecond lower restraint.

Note that, with the processing example 2 as well, the amount of data tobe transferred in the above-mentioned step S202 can be reduced byimposing the above-mentioned upper restraint.

FIG. 13 is a diagram for describing a reduction effect of the amount ofdata to be transferred in the event that the adaptive filter 162 whichperforms the processing of the processing example 2 imposes the upperrestraint on the reference region R.

(a) in FIG. 13 is a diagram illustrating regions of pixels to betransferred in the adaptive filter processing in the event of imposingneither the upper restraint nor the first lower restraint. Here, asillustrated in (a) in FIG. 13, with the processing example 2, the targetLCU and ALFU2 corresponding to the target LCU are set to mutuallydifferent positions.

As illustrated in (a) in FIG. 13, in the event that neither the upperrestraint nor the first lower restraint are imposed, there have to betransferred the pixel value of a pixel positioned above the target ALFU2and the pixel value of a pixel positioned below the target ALFU2 whichare pixels in the vicinity of the target ALFU2.

(b) in FIG. 13 is a diagram illustrating regions of pixels to betransferred in the adaptive filter processing in the event that theadaptive filter 162 which performs the processing of the processingexample 2 imposing the upper restraint. As illustrated in (b) in FIG.13, in the event of imposing the upper restraint, the adaptive filterprocessing can be performed without transferring the pixel value of apixel positioned above the target ALFU2. Note that (c) in FIG. 13illustrates a reference range of the deblocking filter in the LCU.

(Mathematical Expressions of Upper Restraint, First Lower Restraint, andSecond Lower Restraint)

Hereinafter, description will be made regarding mathematical expressionsof the upper restraint, first lower restraint, and second lowerrestraint, with reference to FIG. 14.

(a) in FIG. 14 is a diagram for describing restraints to be imposed onthe upward reference range DDupper (yy) in the event that the loopfilter 161 performs the processing illustrated in the processingexample 1. As illustrated in (a) in FIG. 14, in the event thatcoordinates within the ALFU1 of the target pixel are (xx, yy), the Ycoordinate within the ALFU1 of the upper edge of the reference region Ris expressed as yy−DDupper(yy). The upper restraint can be expressedsuch that the Y coordinate within the ALFU1 of the upper edge of thereference region R is equal to or greater than 0. Accordingly, the upperrestraint can be expressed as follows.

DDupper(yy)≤yy  Upper restraint:

Hereinafter, the upper limit value of the DDupper(yy) that satisfies theupper restraint will also be expressed as an upward referable rangeMaxDDupper(yy). Here, the following relation holds.

MaxDDupper(yy)=yy

(b) in FIG. 14 is a diagram for describing restraints to be imposedregarding the downward reference range DDlower(yy) in the event that theloop filter 161 performs the processing illustrated in the processingexample 1. As illustrated in (b) in FIG. 14, in the event thatcoordinates within the ALFU1 of the target pixel are (xx, yy), the Ycoordinate within the ALFU1 of the lower edge of the reference region Ris expressed as yy+DDlower(yy). The above-mentioned first lowerrestraint can be expressed such that the Y coordinate within the ALFU1of the lower edge of the reference region R is equal to or smaller thanLLCUy−1. Accordingly, the first lower restraint can be expressed asfollows.

First lower restraint: DDlower(yy)≤LLCUy−yy−1

Hereinafter, the upper limit value of the DDlower(yy) that satisfies thefirst lower restraint will also be expressed as a first downwardreferable range MaxDDlower1(yy). Here, the following relation holds.

MaxDDlower1(yy)=LLCUy−yy−1

(c) in FIG. 14 is a diagram for describing restraints to be imposed onthe downward reference range DDlower(yy) in the event that the loopfilter 161 performs the processing illustrated in the processing example2. As illustrated in (c) in FIG. 14, in the event that coordinateswithin the ALFU2 of the target pixel are (xx, yy), the Y coordinatewithin the LCU of the lower edge of the reference region R is expressedas yy−Ly+DDlower(yy). The above-mentioned second lower restraint can beexpressed such that the Y coordinate within the LCU of the lower edge ofthe reference region R is equal to or smaller than LLCUy−1−Ddf. Here,Ddf represents width in the Y direction of a region where deblocking inthe target LCU has not been completed. Accordingly, the second lowerrestraint can be expressed as follows.

DDlower(yy)≤LLCUy+Ly−Ddf−yy−1  Second lower restraint:

Hereinafter, the upper limit value of the DDlower(yy) that satisfies thesecond lower restraint may also be expressed as a second downwardreferable range MaxDDlower2 (yy). Here, the following relation holds.

MaxDDlower2(yy)=LLCUy+Ly−Ddf−yy−1

(d) in FIG. 14 illustrates MaxDDupper(yy), MaxDDlower1(yy), andMaxDDlower2(yy) in the event that coordinates within the ALFU2 of thetarget pixel are (xx, yy).

Note that the upward referable range MaxDDupper, first downwardreferable range MaxDDlower1, and second downward referable rangeMaxDDlower2 will collectively be referred to as a referable range.

FIG. 15 is a table illustrating a specific example of the Y coordinatewithin the ALFU, Y coordinate within the LCU, normalized Y coordinatewithin the LCU, upward referable range MaxDDupper, first downwardreferable range MaxDDlower1, and second downward referable rangeMaxDDlower2. As illustrated in FIG. 15, the referable range depends onLy that represents positional deviation between the target LCU and thetarget ALFU. Here, the virtual boundary line VBLine (=LcuHeight−Ly) canbe defined from the Ly that represents positional deviation between thetarget LCU and the target ALFU. With the table in FIG. 15, the virtualboundary line VBLine is equivalent to a case where normalized Ycoordinate yr within the LCU is (0−Ly) % LLCU. In FIG. 15, in the eventthat the normalized Y coordinate yr within the LCU is equal to orsmaller than the virtual boundary line VBLine, the upward referablerange MaxDDupper is specifically, with (0−Ly) % LLCU, (1−Ly) % LLCU,(2−Ly) % LLCU, and (3−Ly) % LLCU, restricted to 0, 1, 2, and 3. This isequivalent to a case where difference between the virtual boundary lineVBLine and the normalized Y coordinate yr within the LCU dist2VB=(ycoordinate % LcuHeight)−VBLine is 0, 1, 2, and 3. The restraint of thisreference region R is as already described in the restraint C.

Also, in the event that the normalized Y coordinate yr within the LCU isless than the virtual boundary line VBLine, the first downward referablerange MaxDDlower1 is specifically, with (LLCU−1−Ly) % LLCUy, (LLCU−2−Ly)% LLCUy, (LLCU−3−Ly) % LLCUy, and (LLCU−4−Ly) % LLCUy, restricted to 0,1, 2, and 3. This is equivalent to a case where difference between thevirtual boundary line VBLine and the normalized Y coordinate yr withinthe LCU dist2VB is −1, −2, −3, and −4. The restraint of this referenceregion R is as already described in the restraint E.

(Of Referable Range, Expressions Using Coordinates within LCU)

Hereinafter, of the referable range, expressions using coordinateswithin the LCU will be described. With the first filter processing unitALFU1, a relation between the Y coordinate yy within the ALFU1 and the Ycoordinate y within the LCU is given by

y=yy

and accordingly, if we express the upward referable range and firstdownward referable range as MaxDupper_1(y) and MaxDlower1_1(y)respectively in the event that the Y coordinate within the LCU of thetarget pixel is y, the following holds.

MaxDupper_1(y)=y  (A)

MaxDlower1_1(y)=LLCUy−y−1  (B)

On the other hand, with the second filter processing unit ALFU2,relations between the Y coordinate yy within the ALFU2 and the Ycoordinate y within the LCU are given by the following.

y=yy+LLCUy−Ly (0≤yy≤Ly−1)

y=yy−Ly (Ly≤yy≤LLCUy−1)

These relations can also be expressed as follows.

yy=y+Ly (0≤y≤LLCUy−Ly−1)

yy=y+Ly−LLCUy (LLCUy−Ly≤y≤LLCUy−1)

Accordingly, if we express the upward referable range, first downwardreferable range, and second downward referable range as MaxDupper_2(y),MaxDlower1_2(y) and MaxDlower2_2(y) respectively in the event that the Ycoordinate within the LCU of the target pixel is y,

$\begin{matrix}{\mspace{85mu} \left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 2} \right\rbrack} & \; \\{{{MaxDupper\_}2(y)} = \left\{ \begin{matrix}{y + L_{y}} & \left( {0 \leq y \leq {{LLCU}_{y} - L_{y} - 1}} \right) \\{y + L_{y} - {LLCU}_{y}} & \left( {{{LLCU}_{y} - L_{y}} \leq y \leq {{LLCU}_{y} - 1}} \right)\end{matrix} \right.} & (C) \\{\mspace{79mu} \left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 3} \right\rbrack} & \; \\{{{MaxDlower2\_}2(y)} = \left\{ \begin{matrix}{{LLCU}_{y} - {Ddf} - y - 1} & \left( {0 \leq y \leq {{LLCU}_{y} - L_{y} - 1}} \right) \\{{2 \times {LLCU}_{y}} - {Ddf} - y - 1} & \left( {{{LLCU}_{y} - L_{y}} \leq y \leq {{LLCU}_{y} - 1}} \right)\end{matrix} \right.} & (D) \\{\mspace{79mu} \left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 4} \right\rbrack} & \; \\{{{MaxDlower1\_}2(y)} = \left\{ \begin{matrix}{{LLCU}_{y} - L_{y} - y - 1} & \left( {0 \leq y \leq {{LLCU}_{y} - L_{y} - 1}} \right) \\{{2 \times {LLCU}_{y}} - L_{y} - y - 1} & \left( {{{LLCU}_{y} - L_{y}} \leq y \leq {{LLCU}_{y} - 1}} \right)\end{matrix} \right.} & (E)\end{matrix}$

hold.

The mathematical expressions (A), (B), (C), (D), and (E) correspond tothe above-mentioned restraint A, restraint B, restraint C, restraint D,and restraint E, respectively. The restraint A and restraint Ccorrespond to the upper restraint regarding the first filter processingunit ALFU1, and the second filter processing unit ALFU2, the restraint Band restraint E correspond to the first lower condition regarding thefirst filter processing unit ALFU1, and the second filter processingunit ALFU2, and the restraint D corresponds to the second lowercondition regarding the second filter processing unit ALFU2.

Now, let us say that the virtual boundary line VBLine is LLCUy−Ly(=LcuHeight−Ly), the mathematical expression (C) corresponding to therestraint C means, in the event that y that indicates the Y coordinatewithin the LCU is positioned equal to or lower than the virtual boundaryline VBLine, that is, in the event of (LLCUy−Ly≤y≤LLCUy−1), to restrictthe upward reference range MaxDupper_2(y) to difference between the Ycoordinate within the LCU and virtual boundary line (=y+Ly−LLCUy). Thedifference between the Y coordinate within the LCU and virtual boundaryline (=y+Ly−LLCUy) is equivalent to already described dist2VB=(ycoordinate % LcuHeight)−VBLine. When the number of taps is nine taps,that is, in the event that the upward reference range when there is norestraint is 4, in the case that the upward reference rangeMaxDupper_2(y) is less than 4, the upward reference range is restricted.That is to say, the upward reference range is restricted in the eventthat difference between the Y coordinate within the LCU and the virtualboundary line dist2VB is 0, 1, 2, or 3. When the number of taps is seventaps (the upward reference range at the time of no restraint is three),in the event that the difference between the Y coordinate within the LCUand the virtual boundary line dist2VB is 0, 1, 2, or 3, the upwardreference range is restricted. When the number of taps is five taps (theupward reference range at the time of no restraint is two), in the eventthat the difference between the Y coordinate within the LCU and thevirtual boundary line dist2VB is 0 or 1, the upward reference range isrestricted.

The mathematical expression (E) corresponding to the restraint E means,in the event that y that indicates the Y coordinate within the LCU ispositioned above the virtual boundary line VBLine (=LcuHeight−Ly), thatis, in the event of (0≤y≤LLCUy−1), to restrict the downward referencerange MaxDlower2_2(y) to difference between the Y coordinate within theLCU and virtual boundary line (=VBLine−y)−1. The difference between theY coordinate within the LCU and virtual boundary line (=VBLine−y) isequivalent to a value obtained by inverting the sign of alreadydescribed dist2VB=(y coordinate % LcuHeight)−VBLine. When the number oftaps is nine taps, that is, in the event that the upward reference rangewhen there is no restraint is 4, in the case that the −dist2VB−1 becomesless than 4, the downward reference range is restricted. That is to say,the downward reference range is restricted in the event that differencebetween the Y coordinate within the LCU and the virtual boundary linedist2VB is −1, −2, −3, or −4. When the number of taps is seven taps (thedownward reference range at the time of no restraint is three), in theevent that the difference between the Y coordinate within the LCU andthe virtual boundary line dist2VB is −1, −2, or −3, the downwardreference range is restricted. When the number of taps is five taps (thedownward reference range at the time of no restraint is two), in theevent that the difference between the Y coordinate within the LCU andthe virtual boundary line dist2VB is −1 or −2, the downward referencerange is restricted.

(Restraint A)

The mathematical expression (A) is satisfied in the event that the loopfilter 16 performs any processing of the processing example 1 andprocessing example 2, and when the Y coordinate within the LCU of thetarget pixel is less than a predetermined threshold, by restricting theupward reference range Dupper(y) to equal to or smaller than the Ycoordinate within the LCU of the target pixel. In the event ofperforming any processing of the processing example 1 and processingexample 2, the loop filter 16 sets the reference region R so as tosatisfy the following restraint A.

Restraint A: The upward reference range Dupper(y) is set to the numberof pixels equal to smaller than the Y coordinate within the LCU of thetarget pixel in the event that the Y coordinate within the LCU of thetarget pixel is less than a predetermined threshold Lupper1.

Here, as the predetermined threshold Lupper1, a fixed value such as 4 or8 or the like may be employed, or a variable value may be employed (thiswill be applied to the following). Also, the upward reference rangeDupper (y) in the event that the Y coordinate within the LCU of thetarget pixel is less than the predetermined threshold Lupper1 may be setso as to depend on the Y coordinate within the LCU of the target pixel,or may be set to, for example, 0 without depending on the Y coordinatewithin the LCU of the target pixel.

(Restraint B)

The mathematical expression (B) is satisfied in the event that the loopfilter 16 performs any processing of the processing example 1 andprocessing example 2, and when the Y coordinate within the LCU of thetarget pixel is equal to or greater than a predetermined threshold, byrestricting the downward reference range Dlower(y) to equal to orsmaller than a value obtained by subtracting the Y coordinate within theLCU of the target pixel from the LLCUy and adding one thereto. In theevent of performing any processing of the processing example 1 andprocessing example 2, the loop filter 16 sets the reference region R soas to satisfy the following restraint B.

Restraint B: The downward reference range Dlower(y) is set to the numberof pixels equal to or smaller than LLCUy−(Y coordinate within the LCU ofthe target pixel) in the event that the Y coordinate within the LCU ofthe target pixel is equal to or greater than a predetermined thresholdLLCUy−Lupper1.

Here, the downward reference range Dlower(y) in the event that the Ycoordinate within the LCU of the target pixel is equal to or greaterthan the predetermined threshold LLCUy−Lupper1 may be set so as todepend on the Y coordinate within the LCU of the target pixel, or may beset to, for example, 0 without depending on the Y coordinate within theLCU of the target pixel.

(Restraint C)

The mathematical expression (C) is satisfied in the event that the loopfilter 16 performs the processing of the processing example 2, and whenthe normalized Y coordinate within the LCU of the target pixel is equalto or greater than a predetermined threshold, by restricting the upwardreference range Dupper(y) to equal to or smaller than Ly+(Y coordinatewithin the LCU of the target pixel)−LLCUy. In the event of performingthe processing of the processing example 2, the loop filter 16 sets thereference region R so as to satisfy the following restraint C.

Restraint C: The upward reference range Dupper(y) is set to the numberof pixels equal to or smaller than Ly+(normalized Y coordinate withinthe LCU of the target pixel)−LLCUy in the event that the normalized Ycoordinate within the LCU of the target pixel is equal to or greaterthan a predetermined threshold Lupper2.

Here, the predetermined threshold Lupper2 has to be a value equal to orsmaller than LLCUy−Ly (this will be applied to the following). Also, theupward reference range Dupper(y) in the event that the normalized Ycoordinate within the LCU of the target pixel is equal to or greaterthan the predetermined threshold Lupper2 may be set so as to depend onthe Y coordinate within the LCU of the target pixel, or may be set to,for example, 0 without depending on the Y coordinate within the LCU ofthe target pixel.

(Restraint D)

The mathematical expression (D) is satisfied in the event that the loopfilter 16 performs the processing of the processing example 2, and whenthe normalized Y coordinate within the LCU of the target pixel is lessthan a predetermined threshold, by restricting the downward referencerange Dlower(y) to equal to or smaller than LLCUy−Ddf−(normalized Ycoordinate within the LCU of the target pixel)−1. In the event ofperforming the processing of the processing example 2, the loop filter16 sets the reference region R so as to satisfy the following restraintD.

Restraint D: The downward reference range Dlower(y) is set to the numberof pixels equal to or smaller than LLCUy−Ddf−(normalized Y coordinatewithin the LCU of the target pixel)−1 in the event that the normalized Ycoordinate within the LCU of the target pixel is less than apredetermined threshold Llower2.

Here, the downward reference range Dlower(y) in the event that thenormalized Y coordinate within the LCU of the target pixel is less thanthe predetermined threshold Llower2 may be set so as to depend on the Ycoordinate within the LCU of the target pixel, or may be set to, forexample, 0 without depending on the Y coordinate within the LCU of thetarget pixel. Also, the threshold Llower2 has to be a value equal to orgreater than LLCUy−Ly.

(Restraint E)

The mathematical expression (E) is satisfied in the event that the loopfilter 16 performs the processing of the processing example 2, and whenthe normalized Y coordinate within the LCU of the target pixel is lessthan a predetermined threshold, by restricting the downward referencerange Dlower(y) to equal to or smaller than LLCUy−Ly−(normalized Ycoordinate within the LCU of the target pixel)−1. In the event ofperforming the processing of the processing example 2, the loop filter16 sets the reference region R so as to satisfy the following restraintE.

Restraint E: The downward reference range Dlower(y) is set to the numberof pixels equal to or smaller than LLCUy−Ly−(normalized Y coordinatewithin the LCU of the target pixel)−1 in the event that the normalized Ycoordinate within the LCU of the target pixel is less than apredetermined threshold Llower3.

Here, the threshold Llower3 has to be a value equal to or greater thanLLCUy−Ly.

(Specific Example of Referable Range)

FIG. 16 illustrates a specific example of the Y coordinate y within theLCU in the event that LLCUy=32 and Ly=0, the upward referable rangeMaxDupper_1(y), and first downward referable range MaxDlower1_1(y). Asillustrated in FIG. 16, the smaller the Y coordinate within the LCU, thesmaller the upward referable range MaxDupper_1(y). Also, the greater theY coordinate within the LCU, the smaller the first downward referablerange MaxDlower1_1(y).

As an example, let us consider a case where the number of taps that thetap count information included in the filter parameters FP is seven. Inthis case, when there is no restraint, the upward referable range anddownward referable range in the reference region R are both threepixels. Accordingly, in the event that the upward referable rangeMaxDupper_1(y) goes into equal to or smaller than three pixels (in theevent that the Y coordinate within the LCU is less than four), theupward reference range is restricted by the above-mentioned restraint A,whereby filter processing can be performed without referencing a pixelvalue on the upper side of the target LCU (target ALFU1). Also, in theevent that the downward referable range MaxDlower1_1(y) goes into equalto or smaller than three pixels (in the event that the Y coordinatewithin the LCU is less than 32−4=28), the downward reference range isrestricted by the above-mentioned restraint B, whereby filter processingcan be performed without referencing a pixel value on the lower side ofthe target LCU (target ALFU1).

FIG. 17 illustrates a specific example of the Y coordinate within theALFU, Y coordinate within the LCU, and normalized Y coordinate withinthe LCU in the event that LLCUy=32 and Ly=8, the upward referable rangeMaxDDupper_2(yy), first downward referable range MaxDlower1(yy), andsecond downward referable range MaxDlower2(yy). Also, FIG. 18illustrates a specific example of the normalized Y coordinate within theLCU in the event that LLCUy=32 and Ly=8, the upward referable rangeMaxDDupper_2(y), first downward referable range MaxDlower1_2(y), andsecond downward referable range MaxDlower2_2(y).

With the example illustrated in FIG. 18, in the event that thenormalized Y coordinate within the LCU is equal to or greater than 24,the upward reference range is restricted by the above-mentionedrestraint C. Also, in the event that the normalized Y coordinate withinthe LCU is less than 24, the second upward reference range is restrictedby the above-mentioned restraint D, and the first downward referencerange is restricted by the above-mentioned restraint E.

(Example of Another Restraint to be Imposed on Reference Region)

Description has been made so far regarding a configuration wherein thereference region setting unit 162 b included in the loop filter 16restricts at least one of the upward reference range and downwardreference range, but the present embodiment is not restricted to this.For example, the reference region setting unit 162 b may be configuredto set the reference region R by imposing at least one of the followingrestraints F to H in addition to at least one restraint of theabove-mentioned restraints A to E. Also, the reference region settingunit 162 b may be configured to set the reference region R by imposingat least one of the following restraints F to H without imposing any ofthe above-mentioned restraints A to E.

(Restraint F)

In the event that the loop filter 16 performs any processing of theprocessing example 1 and processing example 2, the reference regionsetting unit 162 b sets the reference region R so as to satisfy thefollowing restraint F.

Restraint F: A rightward reference range Dright is set to the number ofpixels equal to or smaller than LLCUx−(X coordinate within the LCU ofthe target pixel)−1 in the event that the X coordinate within the LCU ofthe target pixel is equal to or greater than a predetermined thresholdLLCUx−Lright1.

The rightward reference range Dright mentioned here specifies distancealong the X axis between the target pixel and a pixel within thereference region R which is a pixel adjacent to the right edge of thereference region R (this will be applied to the following). Also, therightward reference range Dright is represented with a pixel as a unit(this will be applied to the following).

Also, the rightward reference range Dright (x) in the event that the Xcoordinate within the LCU of the target pixel is equal to or greaterthan a predetermined threshold LLCUx−Lright1 may be set so as to dependon the X coordinate within the LCU of the target pixel, or may be setto, for example, 0 without depending on the X coordinate within the LCUof the target pixel.

The reference region setting unit 162 b sets the reference region Rwhile imposing the restraint F thereon, and accordingly, the filterprocessing unit 162 c generates the adaptive-filtered decoded imageP_ALF without referencing a pixel value on the right side of the filterprocessing unit ALFU. Accordingly, the amount of data to be transferredfrom the frame memory 15 to the internal memory 163, and the amount ofdata to be transferred from the internal memory 163 to the filterprocessing unit 162 c are reduced. Also, the filter processing unit 162c can perform filter processing without awaiting until decoding of anLCU on the right side of the target LCU is completed, and accordingly,the processing time is reduced.

(Restraint G)

In the event that the loop filter 16 performs the processing of theprocessing example 2, the reference region setting unit 162 b sets thereference region R so as to satisfy the following restraint G.

Restraint G: A rightward reference range Dright is set to the number ofpixels equal to or smaller than LLCUx−Ddf−(normalized X coordinatewithin the LCU of the target pixel)−1 in the event that the normalized Xcoordinate within the LCU of the target pixel is less than apredetermined threshold Lright2.

The normalized X coordinate within the LCU mentioned here is definedwith (X coordinate within the LCU) % LLCUx in which a symbol “%” thatrepresents remainder operation is used, and takes a value equal to orgreater than 0 (this will be applied to the following). Also, let us saythat Ddf represents width in the X direction of a region wheredeblocking by the deblocking filter 161 has not been completed in thetarget LCU.

Also, the rightward reference range Dright in the event that thenormalized X coordinate within the LCU of the target pixel is less thana predetermined threshold Lright2 may be set so as to depend on the Xcoordinate within the LCU of the target pixel, or may be set to, forexample, 0 without depending on the X coordinate within the LCU of thetarget pixel. Also, the threshold Lright2 has to be a value equal to orgreater than LLCUx−Lx.

The reference region setting unit 162 b sets the reference region Rwhile imposing the restraint F thereon, whereby the amount of data to betransferred from the frame memory 15 to the internal memory 163 can bereduced. Also, the filter processing unit 162 c can perform filterprocessing without awaiting until deblocking regarding the right edge ofthe target LCU is completed, and accordingly, the processing time isreduced.

Note that the reference region setting unit 162 b may be configured toset the reference region R so that the rightward reference region Drightsatisfies Dright LLCUx−Ddf−(normalized X coordinate within the LCU ofthe target pixel)−1 in the event that the normalized X coordinate withinthe LCU is equal to or greater than a predetermined threshold Lright2′and also less than a predetermined threshold Lright2. Here, it isdesirable that the threshold Lright2′ satisfies Lright2′=LLCUx−Lx−Dtap.

(Restraint H)

In the event that the loop filter 16 performs the processing of theprocessing example 2, the reference region setting unit 162 b sets thereference region R so as to satisfy the following restraint H.

Restraint H: The rightward reference range Dright is set to the numberof pixels equal to or smaller than LLCUx−Lx−(normalized X coordinatewithin the LCU of the target pixel)−1 in the event that the normalized Xcoordinate within the LCU of the target pixel is less than apredetermined threshold Lright3.

Here, the threshold Lright3 has to be a value equal to or greater thanLLCUx−Lx.

FIG. 19 illustrates an example of the reference region R set so as tosatisfy both the of the restraint D and restraint G.

(Moving Image Encoding Device 2)

Description will be made regarding the configuration of the moving imageencoding device 2 according to the present embodiment, with reference toFIG. 20 to FIG. 21. The moving image encoding device 2 is an encodingdevice which includes H.264/MPEG-4.AVC, technology employed by KTAsoftware which is a codec for joint development in VCEG (Video CodingExpert Group), and technology employed by TMuC (Test Model underConsideration) software which is a succeeding codec thereof at a portionthereof. Hereinafter, the same portion as with an already describedportion is denoted with the same reference numeral, and descriptionthereof will be omitted.

FIG. 20 is a block diagram illustrating the configuration of the movingimage encoding device 2. As illustrated in FIG. 20, the moving imageencoding device 2 includes a predicted image generator 21, atransformation/quantization unit 22, an inverse quantization/inversetransformation unit 23, an adder 24, frame memory 25, a loop filter 26,a variable length code encoding unit 27, and a subtractor 28. Also, asillustrated in FIG. 20, the predicted image generator 21 includes anintra predicted image generator 21 a, a motion vector detecting unit 21b, an inter predicted image generator 21 c, a prediction procedurecontrol unit 21 d, and a motion vector redundancy removing unit 21 e.The moving image encoding device 2 is a device configured to generateencoded data #1 by encoding a moving image #10 (image to be encoded).

(Predicted Image Generator 21)

The predicted image generator 21 recursively divides the LCU to beprocessed into one or multiple lower-order CUs, further divides eachleaf CU into one or multiple partitions, and generates an interpredicted image Pred_Inter using inter-frame prediction or an intrapredicted image Pred_Intra using intra-screen prediction for eachpartition. The generated inter predicted image Pred_Inter and intrapredicted image Pred_Intra are supplied to the adder 24 and subtractor28 as a predicted image Pred.

Note that the predicted image generator 21 omits, with regard to a PU towhich the skip mode has been applied, encoding of another parameterbelonging to this PU. Also, the following is determined so as tooptimize coding efficiency: (1) Mode for dividing the target LCU intolower-order CUs and partitions, (2) Whether to apply the skip mode, and(3) Which of an inter predicted image Pred_Inter, and intra predictedimage Pred_Intra is generated for each partition.

(Intra Predicted Image Generator 21 a)

The intra predicted image generator 21 a generates a predicted imagePred_Intra regarding each partition by intra-screen prediction.Specifically, (1) A prediction mode to be used for intra prediction isselected regarding each partition, and (2) The predicted imagePred_Intra is generated from the decoded image P using the selectedprediction mode. The intra predicted image generator 21 a supplies thegenerated intra predicted image Pred_Intra to the prediction procedurecontrol unit 21 d.

Also, the intra predicted image generator 21 a determines an estimatedprediction mode for the target partition from the prediction modeassigned to a partition in the vicinity of the target partition,supplies a flag MPM that indicates whether or not this estimatedprediction mode agrees with the prediction mode actually selectedregarding the target partition to the variable length code encoding unit27 via the prediction procedure control unit 21 d as a part of the intraprediction parameters PP_Intra, and the variable length code encodingunit 27 includes this flag in the encoded data #1.

Also, in the event that the estimated prediction mode regarding thetarget partition differs from the prediction mode actually selectedregarding the target partition, the intra predicted image generator 21 asupplies a residual prediction mode index RIPM that indicates theprediction mode regarding the target partition to the variable lengthcode encoding unit 27 via the prediction procedure control unit 21 d asa part of the intra prediction parameters PP_Intra, and the variablelength code encoding unit 27 includes this residual prediction modeindex in the encoded data #1.

(Motion Vector Detecting Unit 21 b)

The motion vector detecting unit 21 b detects a motion vector myrelating to each partition. Specifically, the motion vector detectingunit 21 b detects the motion vector my relating to the target partitionby (1) selecting the adaptive-filtered decoded image P_ALF′ to be usedas a reference image, and (2) searching for a region most approximate tothe target partition in the selected adaptive-filtered decoded imageP_ALF′. Here, the adaptive-filtered decoded image P_ALF′ is an image tobe obtained by subjecting a decoded image where decoding of the entireframe has already been completed to adaptive filter processing by theloop filter 26, and the motion vector detecting unit 21 b can read outthe pixel value of a pixel which makes up the adaptive-filtered decodedimage P_ALF′ from the frame memory 25. The motion vector detecting unit21 b supplies the detected motion vector my to the inter predicted imagegenerator 21 c and motion vector redundancy removing unit 21 e alongwith the reference image index RI for specifying the adaptive-filtereddecoded image P_ALF′ used as a reference image.

(Inter Predicted Image Generator 21 c)

The inter predicted image generator 21 c generates a motion compensationimage mc regarding each inter prediction partition by inter-frameprediction. Specifically, the inter predicted image generator 21 cgenerates a motion compensation image mc from the adaptive-filtereddecoded image P_ALF′ specified by the reference image index RI suppliedfrom the motion vector detecting unit 21 b using the motion vector mysupplied from the motion vector detecting unit 21 b. In the same way aswith the motion vector detecting unit 21 b, the inter predicted imagegenerator 21 c can read out the pixel value of a pixel which makes upthe adaptive-filtered decoded image P_ALF′ from the frame memory 25. Theinter predicted image generator 21 c supplies the generated motioncompensation image mc (inter predicted image Pred_Inter) to theprediction procedure control unit 21 d along with the reference imageindex RI supplied from the motion vector detecting unit 21 b.

(Prediction Procedure Control Unit 21 d)

The prediction procedure control unit 21 d compares the intra predictedimage Pred_Intra and inter predicted image Pred_Inter with the image tobe encoded, and selects whether to perform intra prediction or interprediction. In the event of having selected intra prediction, theprediction procedure control unit 21 d supplies the intra predictedimage Pred_Intra to the adder 24 and subtractor 28 as the predictedimage Pred, and also supplies the intra prediction parameters PP_Intrato be supplied from the intra predicted image generator 21 a to thevariable length code encoding unit 27. On the other hand, in the eventof having selected inter prediction, the prediction procedure controlunit 21 d supplies the inter predicted image Pred_Inter to the adder 24and subtractor 28 as the predicted image Pred, and also supplies thereference image index RI, and the estimated motion vector index PMVI andmotion vector residual MVD supplied from a later-described motion vectorredundancy removing unit 21 e to the variable length code encoding unit27 as the inter prediction parameters PP_Inter.

(Motion Vector Redundancy Removing Unit 21 e)

The motion vector redundancy removing unit 21 e deletes redundancy inthe motion vector my detected by the motion vector detecting unit 21 b.Specifically, the motion vector redundancy removing unit 21 e generatesa motion vector residual MVD by (1) selecting an estimation method to beused for estimation of the motion vector mv, (2) deriving the estimatedmotion vector pmv in accordance with the selected estimation method, and(3) subtracting the estimated motion vector pmv from the motion vectormv. The motion vector redundancy removing unit 21 e supplies thegenerated motion vector residual MVD to the prediction procedure controlunit 21 d along with the estimated motion vector index PMVI thatindicates the selected estimation method.

(Transformation/Quantization Unit 22)

The transformation/quantization unit 22 (1) subjects a predictedresidual D obtained by subtracting the predicted image Pred from animage to be encoded to DCT transform (Discrete Cosine Transform) foreach block (transform unit), (2) quantizes a DCT coefficient obtained byDCT transform, and (3) supplies a quantized predicted residual QDobtained by quantization to the variable length code encoding unit 27and inverse quantization/inverse transformation unit 23. Note that thetransformation/quantization unit 22 (1) selects a quantization step QPto be used at the time of quantization, for each TU, (2) suppliesquantization parameter difference Δqp that indicates the size of theselected quantization step QP to the variable length code encoding unit28, and (3) supplies the selected quantization step QP to the inversequantization/inverse transformation unit 23. The quantization parameterdifference Δqp mentioned here specifies a difference value to beobtained by subtracting the value of a quantization parameter qp′regarding a TU subjected to DCT transform/quantization immediatelybefore from the value of a quantization parameter qp (e.g., QP=2^(pq/6))regarding a TU to be subjected to DCT transform/quantization.

(Inverse Quantization/Inverse Transformation Unit 23)

The inverse quantization/inverse transformation unit 23 (1) subjects thequantized predicted residual QD to inverse quantization, (2) subjects aDCT coefficient obtained by inverse quantization to inverse DCT(Discrete Cosine Transform), and (3) supplies the predicted residual Dobtained by inverse DCT transform to the adder 24. At the time ofsubjecting the quantized predicted residual QD to inverse quantization,the inverse quantization/inverse transformation unit 23 uses thequantization step QP supplied from the transformation/quantization unit22. Note that the predicted residual D to be output from the inversequantization/inverse transformation unit 23 is obtained by addingquantization error to the predicted residual D input to thetransformation/quantization unit 22, but common naming is employed herefor convenience of description.

(Adder 24)

The adder 24 generates a (local) decoded image P by adding the predictedimage Pred selected by the prediction procedure control unit 21 d to thepredicted residual D generated at the inverse quantization/inversetransformation unit 23. The (local) decoded image P generated at theadder 24 is supplied to the loop filter 26 and also stored in the framememory 25, and used as a reference image in intra prediction.

(Variable Length Code Encoding Unit 27)

The variable length code encoding unit 27 generates the encoded data #1by subjecting (1) the quantized predicted residual QD and Δqp suppliedfrom the transformation/quantization unit 22, (2) the quantizationparameters PP (inter prediction parameters PP_Inter and intra predictionparameters PP_Intra) supplied from the prediction procedure control unit21 d, and (3) the filter parameters FP supplied from the loop filter 26to variable length encoding.

Note that, as a specific encoding system according to the variablelength code encoding unit 27, there is employed CABAC (Context-basedAdaptive Binary Arithmetic Coding) which is one system of arithmeticencoding/decoding or CAVLC (Context-based Adaptive VLC) which is onesystem of non-arithmetic encoding/decoding.

The variable length code encoding unit 27 determines, for each picture,which coding system of CABAC and CAVLC is employed, and performsencoding using the determined coding system, and also includes codingmode information (entropy_coding_mode_flag) for specifying thedetermined coding system in a picture header PH of the encoded data #1.

(Subtractor 28)

The subtractor 28 generates the predicted residual D by subtracting thepredicted image Pred selected at the prediction procedure control unit21 d from an image to be encoded. The predicted residual D generated atthe subtractor 28 is subjected to DCT transform/quantization by thetransformation/quantization unit 22.

(Loop Filter 26)

The loop filter 26 has (1) a function serving as a deblocking filter(DF: Deblocking Filter) configured to perform smoothing (deblocking) ofan image in the vicinity of a block boundary or partition boundary inthe decoded image P, and (2) a function serving as an adaptive filter(ALF: Adaptive Loop Filter) configured to perform adaptive filterprocessing on an image where the deblocking filter has acted, using thefilter parameters FP.

FIG. 21 is a block diagram illustrating the configuration of the loopfilter 26. As illustrated in FIG. 21, the loop filter 26 includes adeblocking filter 161, an adaptive filter 262, and internal memory 163.

The configuration and operation of the loop filter 26 are generally thesame as with the loop filter 16 included in the moving image decodingdevice 1. Hereinafter, only a portion different from the loop filter 16will be described.

(Adaptive Filter 262)

As illustrated in FIG. 21, the adaptive filter 262 includes a targetpixel setting unit 262 a, a reference region setting unit 262 b, and afilter processing unit 262 c. The target pixel setting unit 262 a hasthe same configuration as with the target pixel setting unit 162 aincluded in the moving image decoding device 1, and accordingly,description thereof will be omitted.

The reference region setting unit 262 b has generally the sameconfiguration as with the reference region setting unit 162 b includedin the moving image decoding device 1. However, let us say that thereference region setting unit 262 b determines tap count Ntap set so asto improve coding efficiency, and references this tap count Ntap insteadof “tap count information included in the filter parameters FP” to bereferenced by the reference region setting unit 162 b. Here, the tapcount Ntap represents the previous tap count on which a restraintregarding the reference region is imposed. Also, the tap countinformation which specifies the tap count Ntap is included in theencoded data #1 as a part of the filter parameters FP.

The filter processing unit 262 c determines a filter coefficient a(i, j)and offset o of the filter to act on the deblocked decoded image P_DB soas to reduce error between an image after filtering and an image to beencoded (original image) as small as possible. For example, the filterprocessing unit 262 c determines the filter coefficient a(i, j) andoffset o so as to reduce square error E that the following Expression(2) indicates as small as possible.

$\begin{matrix}\left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 5} \right\rbrack & \; \\{E = {\sum\limits_{x,y}\left\lbrack {{S_{O}\left( {x,y} \right)} - \left( {\sum\limits_{{({i,j})} \in R}{{a\left( {i,j} \right)} \times {S\left( {{x + i},{y + j}} \right)}}} \right) - o} \right\rbrack^{2}}} & (2)\end{matrix}$

Here, S(x, y) represents a pixel value of the deblocked decoded imageP_DB, and S0(x, y) represents a pixel value of an image to be encoded.Also, R represents a region to be referenced for calculating the pixelvalue of the target pixel, and is set by the reference region settingunit 262 b.

The filter processing unit 262 c generates the adaptive-filtered decodedimage P_ALF by performing the filter processing using the thusdetermined filter coefficient a(i, j) and offset o on the deblockeddecoded image P_DB. Also, the filter processing unit 262 c includes thethus determined filter coefficient a(i, j) and offset o in the filterparameters FP. Specific filter processing that the filter processingunit 262 c performs is the same as the filter processing unit 162 c, andaccordingly, description will be omitted here.

Second Embodiment

With the first embodiment, description has been made regarding a casewhere the adaptive filter included in the loop filter 16 acts on adeblocked decoded image, thereby generating the adaptive-filtereddecoded image P_ALF. With the present embodiment, description will bemade regarding a configuration wherein an adaptive filter included aloop filter acts on the deblocked decoded image P_DB, predicted image P,and predicted residual D, thereby generating the adaptive-filtereddecoded image P_ALF. Note that, hereinafter, the same portion as with analready described portion is denoted with the same reference numeral,and description thereof will be omitted.

(Encoded Data #1′)

A filter coefficient group to be multiplied on each pixel of each of thedeblocked decoded image P_DB, predicted image P, and predicted residualD is included in filter parameters FP′ of encoded data #1′ according tothe present embodiment. Other structures of the encoded data #1′ are thesame as the structures of the encoded data #1 according to the firstembodiment.

FIG. 22 is a block diagram illustrating the configuration of a movingimage decoding device 1′ according to the present embodiment. Asillustrated in FIG. 22, the moving image decoding device 1′ includes aloop filter 16′ instead of the loop filter 16 included in the movingimage decoding device 1 according to the first embodiment. Also, inaddition to the decoded image P, the predicted image Pred and predictedresidual D are also supplied to the loop filter 16′. Otherconfigurations are the same as with the moving image decoding device 1.

(Loop Filter 16′)

FIG. 23 is a block diagram illustrating the configuration of the loopfilter 16′. As illustrated in FIG. 23, the loop filter 16′ includes adeblocking filter 161, an adaptive filter 162′, and internal memory 163.Also, the adaptive filter 162′ includes a target pixel setting unit 162a, a reference region setting unit 162 b′, and a filter processing unit162′

With the internal memory 163′, the decoded image P, predicted imagePred, predicted residual D, deblocked decoded image P_DB, andadaptive-filtered decoded image P_AFL are stored.

The loop filter 16′ performs generally the same processing as with anyof the processing example 1 and processing example 2 described in thefirst embodiment. However, the loop filter 16′ references, with theadaptive filter processing, not only the deblocked decoded image P_DBbut also the predicted image Pred and predicted residual P_DB.

(Reference Region Setting Unit 162 b′)

The reference region setting unit 162 b′ sets, in the vicinity of thetarget pixel, a reference region RDB regarding the deblocked decodedimage P_DB, and a reference region RD regarding the predicted residualD, and a reference region RPred regarding the predicted image Pred.

The reference region setting unit 162 b′ sets the reference regions soas to satisfy any restraint of restraints I to restraint M listed below.

Restraint I: Any of the reference region RDB, reference region RD, andreference region RPred is set to 1×1 pixels.

High-speed filtering may be performed by imposing this restraint. Also,with regard to any of the deblocked decoded image P_DB, predicted imagePred, and predicted residual P_DB, the adaptive-filtered decoded imageP_ALF is calculated without referencing a pixel outside the target LCU,and accordingly, the amount of data to be transferred is reduced. Also,the adaptive filter 162′ is configured to reference not only thedeblocked decoded image P_DB but also the predicted image Pred andpredicted residual P_DB, and accordingly, even when setting thereference regions to 1×1 pixels, coding efficiency does notsignificantly deteriorate.

Restraint J: Any of the reference region RDB, reference region RD, andreference region RPred is set so that the number of pixels in thevertical direction becomes one pixel.

High-speed filtering may be performed by imposing this restraint. Also,similarly, with regard to any of the deblocked decoded image P_DB,predicted image Pred, and predicted residual P_DB, the adaptive-filtereddecoded image P_ALF is calculated without referencing pixels on theupper side and lower side of the target LCU, and accordingly, the amountof data to be transferred is reduced. Also, the adaptive filter 162′ isconfigured to reference not only the deblocked decoded image P_DB butalso the predicted image Pred and predicted residual P_DB, andaccordingly, even when setting the number of pixels in the verticaldirection of each reference region is set to one pixel, codingefficiency does not significantly deteriorate.

Restraint K: Of the reference region RDB, reference region RD, andreference region RPred, the reference region RD and reference regionRPred are set to 1×1 pixels.

High-speed filtering may be performed by imposing this restraint. Also,similarly, with regard to any of the predicted image Pred and predictedresidual P_DB, the adaptive-filtered decoded image P_ALF is calculatedwithout referencing a pixel outside the target LCU, and accordingly, theamount of data to be transferred is reduced. Also, no restraint isimposed on the reference region RDB, and accordingly, deterioration incoding efficiency is not brought about.

Restraint L: Of the reference region RDB, reference region RD, andreference region RPred, the reference region RD and reference regionRPred are set so that the number of pixels in the vertical directionbecomes one pixel.

High-speed filtering may be performed by imposing this restraint. Also,similarly, with regard to any of the predicted image Pred and predictedresidual P_DB, the adaptive-filtered decoded image P_ALF is calculatedwithout referencing pixels on the upper side and lower side of thetarget LCU, and accordingly, the amount of data to be transferred isreduced. Also, no restraint is imposed on the reference region RDB, andaccordingly, deterioration in coding efficiency is not brought about.

Restraint M: Of the reference region RDB, reference region RD, andreference region RPred, the reference region RD and reference regionRPred are set so as to satisfy the restraints A to H described in thefirst embodiment.

The amount of data to be transferred is reduced by imposing thisrestraint. Also, no restraint is imposed on the reference region RDB,and accordingly, deterioration in coding efficiency is not broughtabout.

(Filter Processing Unit 162 c′)

The filter processing unit 162 c′ subjects the deblocked decoded imageP_DB, predicted residual D, and predicted image Pred stored in theinternal memory 163′ to filter processing using the filter coefficientgroup included in the filter parameters FP′, thereby generating theadaptive-filtered decoded image P_ALF. More specifically, if we say thata pixel value of the adaptive-filtered decoded image P_ALF isrepresented as SALF(x′, y′), a pixel value of the deblocked decodedimage P_DB is represented as SPDB(x, y), a pixel value of the predictedresidual D is represented as SD(x, y), and a pixel value of thepredicted image Pred is represented as SPred(x, y), the filterprocessing unit 162 c′ calculates the pixel value SALF(x′, y′) by thefollowing Expression (3)

$\begin{matrix}{\mspace{85mu} \left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 6} \right\rbrack} & \; \\{{S_{ALF}\left( {x^{\prime},y^{\prime}} \right)} = {\left( {\sum\limits_{{({u,v})} \in {RDB}}{{a_{PDB}\left( {u,v} \right)} \times {S_{PDB}\left( {{x + u},{y + v}} \right)}}} \right) + \left( {\sum\limits_{{({u,v})} \in {RD}}{{a_{D}\left( {u,v} \right)} \times {S_{D}\left( {{x + u},{y + v}} \right)}}} \right) + \left( {\sum\limits_{{({u,v})} \in {RPred}}{{a_{Pred}\left( {u,v} \right)} \times {S_{Pred}\left( {{x + u},{y + v}} \right)}}} \right) + o}} & (3)\end{matrix}$

Here, the coordinates (x, y) may be the same coordinates as thecoordinates (x′, y′), or may be different coordinates as long as thereis a one-on-one correspondence. Also, aPDB(u, v) represents, of filtercoefficients included in the filter parameters FP′ of the encoded data#1′, a filter coefficient to be multiplied on the pixel value SPDB(x+u,y+v) of the deblocked decoded image P_DB.

Also, aD(u, v) represents, of filter coefficients included in the filterparameters FP′ of the encoded data #1′, a filter coefficient to bemultiplied on the pixel value SD(x+u, y+v) of an image that thepredicted residual D indicates. Also, aPred(u, v) represents, of filtercoefficients included in the filter parameters FP′ of the encoded data#1′, a filter coefficient to be multiplied on the pixel value SPred(x+u,y+v) of an image that the predicted image Pred indicates. Also, aspecific value of the offset o is specified by the filter parametersFP′.

Also, there are employed the reference region RDB, reference region RD,and reference region RPred, which have been set by the reference regionsetting unit 162 b.

Note that the filter processing unit 162 c′ may be configured, in thesame way as with the filter processing unit 162 c in the firstembodiment, to allocate a filter coefficient to each pixel included ineach reference region so as to have a rotational symmetry property of180 degrees.

In this manner, the adaptive filter 162 according to the presentembodiment references the deblocked decoded image P_DB, predictedresidual D, and predicted image Pred, and accordingly, as compared to aconfiguration wherein only the deblocked decoded image P_DB isreferenced, in order to generate a predicted image, a more suitableadaptive-filtered decoded image P_ALF can be generated.

(Moving Image Encoding Device 2′)

Hereinafter, a moving image encoding device 2′ according to the presentembodiment will be described with reference to FIG. 24 to FIG. 25.Hereinafter, the same portion as with an already described portion isdenoted with the same reference numeral, and description thereof will beomitted.

FIG. 24 is a block diagram illustrating the configuration of the movingimage encoding device 2′ according to the present embodiment. Asillustrated in FIG. 24, the moving image encoding device 2′ includes aloop filter 26′ instead of the loop filter 26 of the moving imageencoding device 2 according to the first embodiment. Also, in additionto the decoded image P, the predicted image Pred and predicted residualD are supplied to the loop filter 26′ Other configurations are the sameas with the moving image encoding device 2.

(Loop Filter 26′)

FIG. 25 is a block diagram illustrating the configuration of the loopfilter 26′. As illustrated in FIG. 25, the loop filter 26′ includes adeblocking filter 161, an adaptive filter 262′, and internal memory163′. Also, the adaptive filter 262′ includes a target pixel settingunit 262 a′, a reference region setting unit 262 b′, and a filterprocessing unit 262 c′. The target pixel setting unit 262 a′ andreference region setting unit 262 b′ have the same configuration as thetarget pixel setting unit 162 a′ and reference region setting unit 162b′, and accordingly, description thereof will be omitted here.

The filter processing unit 262 c′ determines a filter coefficientaPDB(u, v) to be multiplied on the pixel value SPDB(x+u, y+v) of thedeblocked decoded image P_DB, a filter coefficient aD(u, v) to bemultiplied on the pixel value SD(x+u, y+v) of an image that thepredicted residual D indicates, a filter coefficient aPred(u, v) to bemultiplied on the pixel value SPred(x+u, y+v) of an image that thepredicted image Pred indicates, and offset o so as to reduce errorbetween an image after filtering and an image to be encoded (originalimage). For example, the filter processing unit 262 c′ determines eachfilter coefficient and offset so as to reduce square error E′ indicatedin the following Expression (4).

$\begin{matrix}{\mspace{85mu} \left\lbrack {{Mathematical}\mspace{14mu} {Expression}\mspace{14mu} 7} \right\rbrack} & \; \\{E^{\prime} = {\sum\limits_{x,y}\begin{bmatrix}{{S_{O}\left( {x,y} \right)} - \left( {\sum\limits_{{({u,v})} \in {RDB}}{{a_{PDB}\left( {u,v} \right)} \times {S_{PDB}\left( {{x + u},{y + v}} \right)}}} \right) -} \\{\left( {\sum\limits_{{({u,v})} \in {RD}}{{a_{D}\left( {u,v} \right)} \times {S_{D}\left( {{x + u},{y + v}} \right)}}} \right) -} \\{\left( {\sum\limits_{{({u,v})} \in {RPred}}{{a_{Pred}\left( {u,v} \right)} \times {S_{Pred}\left( {{x + u},{y + v}} \right)}}} \right) - o}\end{bmatrix}^{2}}} & (4)\end{matrix}$

Here, there are employed the reference region RDB, reference region RD,and reference region RPred, which have been set by the reference regionsetting unit 262 b.

The filter processing unit 262 c′ subjects the deblocked decoded imageP_DB, predicted residual D, and predicted image Pred to filterprocessing using the thus determined filter coefficients and offset,thereby generating the adaptive-filtered decoded image P_ALF. Also, thefilter processing unit 262 c′ includes the thus determined filtercoefficients and offset in the filter parameters FP′. Specific filterprocessing that the filter processing unit 262 c′ performs is the sameas with the filter processing unit 162 c′, and accordingly, descriptionthereof will be omitted here.

Third Embodiment

With the first embodiment and second embodiment, description has beenmade regarding a loop filter wherein a filter reference region is set soas to satisfy various restraints, thereby reducing the amount of data tobe transferred. With the present embodiment, description will be maderegarding a configuration wherein at the time of performing the adaptivefilter processing regarding the target slice, a pixel value iscalculated only regarding the target pixel included in an internalregion set in the inside of this target slice. Hereinafter, the sameportion as with an already described portion is denoted with the samereference numeral, and description thereof will be omitted. Also, theentire configuration of a moving image decoding device and moving imageencoding device according to the present embodiment is the same as theconfiguration of the moving image decoding device 1 and moving imageencoding device 2 according to the first embodiment, and accordingly,description thereof will be omitted. Also, filter parameters to bereferenced by a loop filter according to the present embodiment have thesame structure as with the filter parameters FP according to the firstembodiment.

FIG. 26 is a block diagram illustrating the configuration of a loopfilter 36. As illustrated in FIG. 26, the loop filter 36 includes adeblocking filter 161, an adaptive filter 362, and internal memory 163.Note that, with the present embodiment, the loop filter 36 may have aconfiguration not including internal memory. In the case of such aconfiguration, the deblocking filter 161 reads out each pixel value ofthe decoded image P from the frame memory 15, and stores the deblockeddecoded image P_DB in the frame memory 15. Also, a later-describedfilter processing unit 362 reads out the deblocked decoded image P_DBfrom the frame memory 15 and stores the adaptive-filtered decoded imageP_ALF in the frame memory 15.

As illustrated in FIG. 26, the loop filter 362 includes a target pixelsetting unit 362 a and a filter processing unit 362 c.

The target pixel setting unit 362 a divides the target slice into aninternal region and a peripheral region, and sequentially sets eachpixel included in the internal region as the target pixel. (a) in FIG.27 is a diagram illustrating an internal region R1 and an externalregion R2 set by the target pixel setting unit 362 a. As illustrated in(a) in FIG. 27, a boundary in the vertical direction of the internalregion R1 is set to the inner side from a boundary in the verticaldirection of the target slice by Wx pixels, and a boundary in thehorizontal direction of the internal region R1 is set to the inner sidefrom a boundary in the horizontal direction of the target slice by Wypixels. Here, specific values of the Wx and Wy are set according to thetap count Ntap specified by the tap count information included in thefilter parameters FP. For example, when Ntap=5, Wx=Wy=2, and whenNtap=7, Wx=Wy=3. In general, Wx=Wy=(Ntap−1)/2 holds. Note that when thetap count information is information for independently specifying tapcount Ntapx in the horizontal direction and tap count Ntapy in thevertical direction, Wx=(Ntpx−1)/2, and Wy=(Ntapy−1)/2 hold.

The filter processing unit 362 c calculates a pixel value regarding thetarget pixel set by the target pixel setting unit 362 a. A method forcalculating the target pixel using the filter processing unit 362 c isgenerally the same as with the filter processing unit 162 c in the firstembodiment. However, as the reference region R in the presentembodiment, a region specified by the tap count information included inthe filter parameters FP may be employed without change, or a region onwhich various restraints have been imposed according to the position ofthe target pixel within the LCU as with the first embodiment may beemployed.

(b) in FIG. 27 is a diagram for describing processing by an adaptivefilter according to the related art. With the adaptive filter accordingto the related art, a slice does not have to be divided into an internalregion and a peripheral region, and each pixel included in the slice issequentially set as the target pixel. Accordingly, in the event that thetarget pixel has been set near a boundary of a slice, there has beencaused a case where a filter reference region is set including not onlya pixel within the slice but also a pixel outside the slice (pixelincluded in a region OR in (b) in FIG. 27). The adaptive filteraccording to the related art has generated a pixel value outside a sliceby padding and has used this.

On the other hand, according to the adaptive filter 362 according to thepresent embodiment, a reference region is not set so as to include apixel outside a slice, and accordingly, padding as with the related artdoes not have to be performed. Accordingly, according to the adaptivefilter 362 according to the present embodiment, as compared to theadaptive filter according to the related art, throughput can be reduced.Also, filter processing can be performed without awaiting until eachpixel in the next slice in processing order, which is a slice adjacentto the target slice, goes into referable, and accordingly, processingtime is reduced.

Note that the adaptive filter 362 may also be applied to a moving imageencoding device. In this case, the filter processing unit 362 cdetermines a filter coefficient group so as to reduce error between animage after filtering and an image to be encoded. Specific processing bythe filter processing unit 362 c is generally the same as with thefilter processing unit 162 c, and accordingly, description thereof willbe omitted here.

Fourth Embodiment

With the processing example 2 of the loop filter 16 according to thefirst embodiment, description has been made regarding a case where thesecond filter processing unit ALF2 is set so as to have the same as withthe target LCU, and so as to have a position different from the targetLCU. With the present embodiment, description will be made regarding acase where, with a loop filter configured to perform generally the sameoperation as with the above-mentioned processing example 2, the filterprocessing unit ALFU has the same size as with the target LCU, and isset to the same position as with the target LCU, that is, deblocking andadaptive filter processing are performed for each LCU.

The entire configuration of a moving image decoding device and movingimage encoding device according to the present embodiment is the same asthe configuration of the moving image decoding device 1 and moving imageencoding device 2 according to the first embodiment, and accordingly,description thereof will be omitted. The configuration of a loop filteraccording to the present embodiment is generally the same as with theloop filter 16 according to the first embodiment, but differs in thefollowing point.

Specifically, a reference region setting unit according to the presentembodiment sets, when distance between the target pixel and the rightedge of the target LCU is equal to or smaller than L pixels, thereference region R so that distance between the right edge of thereference region R and the target pixel goes into equal to or smallerthan the L pixels. Also, when distance between the target pixel and thelower edge of the target LCU is equal to or smaller than the L pixels,the reference region setting unit sets the reference region R so thatdistance between the lower edge of the reference region R and the targetpixel goes into equal to or smaller than the L pixels. Here, a specificvalue of the L, a value equal to or greater than the above-mentioned Ddfhas to be employed. FIG. 28 illustrates an example of the referenceregion R set by the reference region setting unit according to thepresent embodiment.

According to such a configuration, even when the adaptive filterperforms processing in increments of LCUs, the amount of data to betransferred can be reduced.

Also, instead of the reference region setting unit performing processingas described above, the target pixel setting unit may be configured toset, as the target pixel, only a pixel where at least either distancefrom the right edge of the target LCU or distance from the lower edge ofthe target LCU is equal to or greater than the L pixel. Also, a specificvalue of the L, a value equal to or greater than the above-mentioned Ddfhas to be employed. According to such a configuration as well, theamount of data to be transferred can be reduced.

(Application)

The above-mentioned moving image encoding devices and moving imagedecoding devices may be used by being mounted on various devicesconfigured to perform transmission, reception, recording, or playing ofa moving image.

First, description will be made wherein the above-mentioned moving imageencoding devices and moving image decoding devices may be used fortransmission and reception of a moving image, with reference to FIG. 30.

(a) in FIG. 30 is a block diagram illustrating the configuration of atransmission device PROD_A on which the moving image encoding device 2has been mounted. As illustrated in (a) in FIG. 30, the transmissiondevice PROD_A includes an encoding unit PROD_A1 configured to obtainencoded data by encoding a moving image, a modulating unit PROD_A2configured to obtain a modulation signal by modulating carrier wavesusing the encoded data obtained by the encoding unit PROD_A1, and atransmission unit PROD_A3 configured to transmit the modulation signalobtained by the modulating unit PROD_A2. The above-mentioned movingimage encoding device 2 is employed as this encoding unit PROD_A1.

The transmission device PROD_A may further include, as a supply sourceof a moving image to be input to the encoding unit PROD_A1, a cameraPROD_A4 configured to image a moving image, a recording medium PROD_A5in which the moving image is recorded, and an input terminal PROD_A6configured to externally input a moving image. In (a) in FIG. 30, aconfiguration is exemplified wherein the transmission device PROD_Aincludes all of these, but a part thereof may be omitted.

Note that the recording medium PROD_A5 may be a medium where a movingimage which has not been encoded is recorded, or may be a medium where amoving image encoded with a coding system for recording different from acoding system for transmission is recorded. In the case of the latter,it is desirable to have a decoding unit (not illustrated) configured todecode encoded data read out from the recording medium PROD_A5 inaccordance with coding system for recording lie between the recordingmedium PROD_A5 and the encoding unit PROD_A1.

(b) in FIG. 30 is a block diagram illustrating the configuration of areception device PROD_B on which the moving image decoding device 1 hasbeen mounted. As illustrated in (b) in FIG. 30, the reception devicePROD_B includes a reception unit PROD_B1 configured to receive amodulation signal, a demodulating unit PROD_B2 configured to obtainencoded data by demodulating the modulation signal received by thereception unit PROD_B1, and a decoding unit PROD_B3 configured to obtaina moving image by decoding the encoded data obtained by the demodulatingunit PROD_B2. The above-mentioned moving image decoding device 1 isemployed as this decoding unit PROD_B3.

The reception device PROD_B may further include, as a supply destinationof a moving image that the decoding unit PROD_B3 outputs, a displayPROD_B4 configured to display a moving image, a recording medium PROD_B5configured to record a moving image, and an output terminal PROD_B6configured to externally output a moving image. In (b) in FIG. 30, aconfiguration is exemplified wherein the reception device PROD_Bincludes all of these, but a part thereof may be omitted.

Note that the recording medium PROD_B5 may be a medium where a movingimage which has not been encoded is recorded, or may be a medium where amoving image encoded with a coding system for recording different from acoding system for transmission is recorded. In the case of the latter,it is desirable to have an encoding unit (not illustrated) configured toencode a moving image obtained from the decoding unit PROD_B3 inaccordance with coding system for recording lie between the decodingunit PROD_B3 and the recording medium PROD_B5.

Note that a transmission medium configured to transmit modulationsignals may be wireless or may be a cable. Also, a transmission mode fortransmitting modulation signals may be broadcasting (here specifies atransmission mode where no transmission destination has not beendetermined beforehand), or may be communication (here specifies atransmission mode where no transmission destination has not beendetermined beforehand). That is to say, transmission of modulationsignals may be realized by any of wireless broadcasting, cablebroadcasting, wireless communication, and cable communication.

For example, a broadcasting station (broadcasting facilities or thelike)/reception station (television receiver or the like) of terrestrialdigital broadcasting is an example of the transmission devicePROD_A/reception device PROD_B which transmits/receives modulationsignals by wireless broadcasting. Also, a broadcasting station(broadcasting facilities or the like)/reception station (televisionreceiver or the like) of cable television broadcasting are an example ofthe transmission device PROD_A/reception device PROD_B whichtransmits/receives modulation signals by cable broadcasting.

Also, a server (workstation or the like)/client (television receiver,personal computer, smart phone, or the like) of a VOD (Video On Demand)service or moving image sharing service or the like using the Internetis an example of the transmission device PROD_A/reception device PROD_Bwhich transmits/receives modulation signals by communication (ingeneral, as for a LAN, any of wireless or cable is employed as atransmission medium, and as for a WAN, cable is employed as atransmission medium). Here, the personal computer also includes adesktop PC, laptop PC, and tablet PC. Also, the smart phone alsoincludes a multi-function mobile phone terminal.

Note that a client of the moving image sharing service has not only afunction for decoding encoded data downloaded from a server to displayon a display but also a function for encoding a moving image imaged by acamera to upload to a server. That is to say, a client of the movingimage sharing service serves as both of the transmission device PROD_Aand reception device PROD_B.

Next, it will be described with reference to FIG. 31 that theabove-mentioned moving image encoding device and moving image decodingdevice can be used for recording and playing of a moving image.

(a) in FIG. 31 is a block diagram illustrating the configuration of arecording device PROD_C on which the above-mentioned moving imageencoding device 2 is mounted. As illustrated in (a) in FIG. 31, therecording device PROD_C includes an encoding unit PROD_C1 configured toobtain encoded data by encoding a moving image, and a writing unitPROD_C2 configured to write the encoded data obtained by the encodingunit PROD_C1 in a recoding medium PROD_M. The above-mentioned movingimage encoding device 2 is employed as this encoding unit PROD_C1.

Note that the recording medium PROD_M may be (1) a type to be housed inthe recording device PROD_C such as an HDD (Hard Disk Drive) or SSD(Solid State Drive) or the like, (2) a type to be connected to therecording device PROD_C such as an SD memory card or USB (UniversalSerial Bus) flash memory or the like, or (3) a type to be loaded to adrive device (not illustrated) housed in the recording device PROD_Csuch as DVD (Digital Versatile Disc) or BD (Blu-ray Disk: registeredtrademark) or the like.

Also, the recording device PROD_C may further include, as a supplysource of a moving image to be input to the encoding unit PROD_C1, acamera PROD_C3 configured to image a moving image, an input terminalPROD_C4 configured to externally input a moving image, and a receptionunit PROD_C5 configured to receive a moving image. In (a) in FIG. 31, aconfiguration is exemplified wherein the recording device PROD_Cincludes all of these, but a part thereof may be omitted.

Note that the reception unit PROD_C5 may be a unit configured to receivea moving image which has not been encoded, or a unit configured toreceive encoded data which has been encoded by a coding system fortransmission different from a coding system for recording. In the caseof the latter, it is desirable to have a decoding unit for transmission(not illustrated) configured to decode encoded data encoded by a codingsystem for transmission lie between the reception unit PROD_C5 and theencoding unit PROD_C1.

Examples of such a recording device PROD_C include a DVD recorder, a BDrecorder, and an HD (Hard Disk) recorder (in this case, the inputterminal PROD_C4 or reception unit PROD_C5 serves a principal supplysource of a moving image). Also, examples of such a recording devicePROD_C include a camcorder (in this case, the camera PROD_C3 serves aprincipal supply source of a moving image), a personal computer (in thiscase, the reception unit PROD_C5 serves a principal supply source of amoving image), and a smart phone (in this case, the camera PROD_C3 orreception unit PROD_C5 serves a principal supply source of a movingimage).

(b) in FIG. 31 is a block illustrating the configuration of a playingdevice PROD_D on which the above-mentioned moving image decoding device1 is mounted. As illustrated in (b) in FIG. 31, the playing devicePROD_D includes a reading unit PROD_D1 configured to read out encodeddata written in the recording medium PROD_M, and a decoding unit PROD_D2configured to obtain a moving image by decoding encoded data read out bythe reading unit PROD_D1. The above-mentioned moving image decodingdevice 1 is employed as this decoding unit PROD_D2.

Note that the recording medium PROD_M may be (1) a type to be housed inthe playing device PROD_D such as an HDD or SSD or the like, (2) a typeto be connected to the playing device PROD_D such as an SD memory cardor USB flash memory or the like, or (3) a type to be loaded to a drivedevice (not illustrated) housed in the playing device PROD_D such as DVDor BD or the like.

Also, the playing device PROD_D may further include, as a supplydestination of a moving image that the decoding unit PROD_D2 outputs, adisplay PROD_D3 configured to display a moving image, an output terminalPROD_D4 configured to externally output a moving image, and atransmission unit PROD_D5 configured to transmit a moving image. In (b)in FIG. 31, a configuration is exemplified wherein the playing devicePROD_D includes all of these, but a part thereof may be omitted.

Note that the transmission unit PROD_D5 may be a unit configured totransmit a moving image which has not been encoded, or a unit configuredto transmit encoded data which has been encoded by a coding system fortransmission different from a coding system for recording. In the caseof the latter, it is desirable to have an encoding unit (notillustrated) configured to encode a moving image using a coding systemfor transmission to be situated between the decoding unit PROD_D2 andthe transmission unit PROD_D5.

Examples of such a playing device PROD_D include a DVD player, a BDplayer, and an HDD player (in this case, the output terminal PROD_D4 tobe connected to a television receiver or the like serves as a principalsupply destination of a moving image). Also, examples of such a playingdevice PROD_D include a television receiver (in this case, the displayPROD_D3 serves a principal supply destination of a moving image), adesktop type PC (in this case, the output terminal PROD_D4 ortransmission unit PROD_D5 serves a principal supply destination of amoving image), a laptop type or tablet type PC (in this case, thedisplay PROD_D3 or transmission unit PROD_D5 serves a principal supplydestination of a moving image), and a smart phone (in this case, thedisplay PROD_D3 or transmission unit PROD_D5 serves a principal supplydestination of a moving image).

(Appendix Matter 1)

The blocks of the above-mentioned moving image decoding device 1, movingimage encoding device 2, moving image decoding device 1′, and movingimage encoding device 2′ may be realized using logic circuits formed onan integrated circuit (IC chip) in a hardware manner, or may be realizedusing a CPU (Central Processing Unit) in a software manner.

In the case of the latter, the above devices include a CPU configured toexecute a command of a program configured to execute the functions, anda storage device (recording medium) such as ROM (Read Only Memory) inwhich this program is stored, RAM (Random Access Memory) configured toload this program, memory in which this program and various types ofdata are stored, and so forth. An object of the present invention may beachieved by supplying a recording medium in which program code(execution type program, intermediate code program, a source program) ofa control program of each of the above-mentioned devices, which issoftware for realizing the above-mentioned functions, is recorded in acomputer-readable manner to each of the above-mentioned devices, and acomputer (or CPU or MPU) thereof reading out and executing the programcode recorded in the recording medium.

Examples of the above-mentioned recording medium include tapes such amagnetic tape, a cassette tape, or the like, disks including a magneticdisk such as a floppy (registered trademark) disk/hard disk or the like,and an optical disc such as CD-ROM/MO/MD/DVD/CD-R or the like, cardssuch as an IC card (including a memory card)/optical card or the like,semiconductor memory such as mask ROM/EPROM/flash ROM or the like, andlogic circuits such as a PLD (Programmable logic device), FPGA (FieldProgrammable Gate Array), or the like.

Also, an arrangement may be made wherein each of the above-mentioneddevices is configured so as to be connected to a communication network,and the above-mentioned program code is supplied via the communicationnetwork. This communication network is not particularly restricted aslong as program code can be transmitted. For example, there areavailable the Internet, intranet, extranet, LAN, ISDN, VAN, CATVcommunication network, virtual private network (Virtual PrivateNetwork), telephone network, mobile communications network, a satellitecommunication network, and so forth. Also, a transmission medium whichmakes up this communication network is also not restricted to a mediumhaving a particular configuration nor a particular type medium as longas program code can be transmitted using a medium. For example, thereare available cable such as IEEE1394, USB, power line carrier, cable TVcircuit, telephone line, ADSL (Asymmetric Digital Subscriber Line)circuit, or the like, infrared radiation such as IrDA or remote control,and wireless such as Bluetooth (registered trademark), IEEE802.11wireless, HDR (High Data Rate), NFC (Near Field Communication), DLNA(Digital Living Network Alliance), portable telephone network, satellitecircuit, terrestrial digital network, or the like.

(Appendix Matter 2)

As described above, an image filter device includes: filter meansconfigured to generate an output image by acting on an input image madeup of a plurality of unit regions; and reference region modifying meansconfigured to reduce a reference region to be referenced for the filtermeans calculating a pixel value of each target pixel to a size accordingto a position of this target pixel in a unit region including thistarget pixel.

In the event that the image filter device is employed for a decodingdevice configured to generate a decoded image by decoding encoded data,or an encoding device configured to generate encoded data by encoding animage to be encoded, a predicted image with high prediction precisioncan be generated by referencing an output image of the image filterdevice, and accordingly, coding efficiency improves.

Also, according to the image filter device thus configured, the filtermeans reduces a reference region to be referenced for calculating apixel value of each target pixel to a size according to the position ofthis target pixel in a unit region including this target pixel, andaccordingly, the number of pixels to be referenced for generating anoutput image can be reduced while maintaining high coding efficiency.Accordingly, according to the image filter device thus configured, theamount of data to be transferred at the time of filter processing can bereduced while maintaining high coding efficiency.

Note that the unit region may be, for example, the largest coding unitLCU, or may be a coding unit CU obtained by dividing the largest codingunit LCU, or may be another region.

Also, it is desirable that the reference region modifying means includefirst setting means configured to set, when distance DU between anupstream edge of a unit region including a target pixel, and this targetpixel is less than a predetermined threshold, a position of an upstreamedge of a reference region regarding this target pixel so as to suppressdistance with this target pixel to the DU or shorter, and second settingmeans configured to set, when distance DL between a downstream edge of aunit region including a target pixel, and this target pixel is less thana predetermined threshold, a position of a downstream edge of areference region regarding this target pixel so as to suppress distancewith this target pixel to the DL or shorter.

The first setting means set, when distance DU between an upstream edgeof a unit region including a target pixel, and this target pixel is lessthan a predetermined threshold, a position of an upstream edge of areference region regarding this target pixel so as to suppress distancewith this target pixel to the DU or shorter, and accordingly, the filtermeans calculate the pixel value of the target pixel without referencingthe pixel value of a pixel positioned on more upstream side than theupstream edge of the unit region. Also, the second setting means set,when distance DL between a downstream edge of a unit region including atarget pixel, and this target pixel is less than a predeterminedthreshold, a position of a downstream edge of a reference regionregarding this target pixel so as to suppress distance with this targetpixel to the DL or shorter, and accordingly, the filter means calculatethe pixel value of the target pixel without referencing the pixel valueof a pixel positioned on more downstream side than the downstream edgeof the unit region. Accordingly, according to the above configuration,the amount of data to be transferred at the time of filter processingcan be reduced.

Also, the first setting means have to set the number of taps of thereference region to a predetermined value when distance DU between anupstream edge of a unit region including a target pixel, and this targetpixel is less than a predetermined threshold, and similarly, the secondsetting means have to set the number of taps of the reference region toa predetermined value when distance DL between a downstream edge of aunit region including a target pixel, and this target pixel is less thana predetermined threshold, and accordingly, high coding efficiency canbe maintained.

Note that the above-mentioned upstream edge specifies one of the upperedge and left edge when the processing is performed in raster scanorder, and specifies one of the lower edge and right edge when theprocessing is performed in inverse order of raster scan. Similarly, theabove-mentioned downstream edge specifies one of the lower edge andright edge when the processing is performed in raster scan order, andspecifies one of the upper edge and left edge when the processing isperformed in inverse order of raster scan (this will be applied to thefollowing).

Also, it is desirable that the image filter device according to thepresent invention further includes smoothing means configured to smootha pixel value of a pixel where of the unit regions, distance from atleast one of two edges on the upstream side in processing order is equalto or shorter than DB, with the filter means generating an output imagefor each output unit to be obtained by shifting each of the plurality ofunit regions on the upstream side in processing order, and with thereference region modifying means including first setting meansconfigured to set, when a target pixel is positioned on the downstreamside of a downstream edge in output unit corresponding to a unit regionincluding this target pixel, a position of an upstream edge of areference region regarding this target pixel so as to suppress distancewith this target pixel to be equal or shorter than distance between thistarget pixel and a downstream edge of this output unit, second settingmeans configured to set, when a target pixel is positioned on theupstream side of a downstream edge in output unit corresponding to aunit region including this target pixel, a position of a downstream edgeof a reference region regarding this target pixel so as to suppressdistance with this target pixel to be equal or shorter than distancebetween this target pixel and a downstream edge of this output unit, orthird setting means configured to set, when a target pixel is positionedon the upstream side of a downstream edge in output unit correspondingto a unit region including this target pixel, a position of a downstreamedge of a reference region regarding this target pixel so as to suppressdistance with this target pixel to be equal or shorter than distance tobe obtained by subtracting the DB from distance between this targetpixel and a downstream edge of this unit region.

The first setting means set, when a target pixel is positioned on thedownstream side of a downstream edge in output unit corresponding to aunit region including this target pixel, a position of an upstream edgeof a reference region regarding this target pixel so as to suppressdistance with this target pixel to be equal or shorter than distancebetween this target pixel and a downstream edge of this output unit, andaccordingly, the filter means calculate the pixel value of the targetpixel without referencing the pixel value of a pixel on more upstreamside than an upstream edge of an output unit including the target pixel.Also, the second setting means set, when a target pixel is positioned onthe upstream side of a downstream edge in output unit corresponding to aunit region including this target pixel, a position of a downstream edgeof a reference region regarding this target pixel so as to suppressdistance with this target pixel to be equal or shorter than distancebetween this target pixel and a downstream edge of this output unit, andaccordingly, the filter means calculate the pixel value of the targetpixel without referencing the pixel value of a pixel on more downstreamside than an downstream edge of an output unit including the targetpixel. Accordingly, according to the above-mentioned configuration, theamount of data to be transferred at the time of filter processing can bereduced.

Also, the third setting means set, when a target pixel is positioned onthe upstream side of a downstream edge in output unit corresponding to aunit region including this target pixel, a position of a downstream edgeof a reference region regarding this target pixel so as to suppressdistance with this target pixel to be equal or shorter than distance tobe obtained by subtracting the DB from distance between this targetpixel and a downstream edge of this unit region, and accordingly, thefilter means calculate the pixel value of the target pixel withoutreferencing a pixel value in a region where smoothing by the smoothingmeans has not been completed. Accordingly, according to theabove-mentioned configuration, the amount of data to be transferred atthe time of filter processing can be reduced, and also filter processingcan be performed without waiting until the smoothing means act on twoedges on the downstream side of the target unit region, and accordingly,processing time can be reduced.

Note that the above “the downstream side of a downstream edge”specifies, in the event that the processing is performed in raster scanorder, one of “the lower side of a lower edge” and “the right side of aright edge”, and in the event that the processing is performed ininverse order of raster scan, specifies one of “the upper side of anupper edge” and “the left side of a left edge”.

Also, the above “the upstream side of a downstream edge” specifies, inthe event that the processing is performed in raster scan order, one of“the upper side of a lower edge” and “the left side of a right edge”,and in the event that the processing is performed in inverse order ofraster scan, specifies one of “the lower side of an upper edge” and “theright side of a left edge”.

Also, it is desirable that the input image is a predicted image.

According to the above-mentioned configuration, the amount of data to betransferred at the time of performing filter processing on a predictedimage can be reduced. Note that, in the event that the above-mentionedimage filter device is employed as a decoding device configured togenerate a decoded image by decoding encoded data, or an encoding deviceconfigured to generate encoded data by encoding an image to be encoded,the above-mentioned predicted image specifies a predicted image to beobtained by referencing a decoded image, for example.

Also, it is desirable that the input image is a residual image.

According to the above-mentioned configuration, the amount of data to betransferred at the time of performing filter processing on a residualimage can be reduced. Note that, in the event that the above-mentionedimage filter device is employed as a decoding device configured togenerate a decoded image by decoding encoded data, the above-mentionedresidual data specifies, for example, residual data to be obtained bydecoding encoded data, and in the event that the above-mentioned imagefilter device is employed as an encoding device configured to generateencoded data by encoding an image to be encoded, the residual imagespecifies a difference image between an image to be encoded and apredicted image.

Also, an image filter device according to the present inventionincludes: filter means configured to generate an output image for eachunit region by acting on an input image made up of a plurality of unitregions; and smoothing means configured to smooth a pixel value of apixel where of the unit regions, distance from at least one of two edgeson the upstream side in processing order is equal to or shorter than DB;with the filter means acting on a target unit region after the smoothingmeans act on two edges on the upstream side of a target unit regionbefore acting on two edges on the upstream side of the next unit regionin processing order.

With the image filter device thus configured, the smoothing means act ona target unit region which the smoothing means have acted on two edgeson the upstream side before acting on two edges on the upstream side ofthe next unit region in processing order. Accordingly, according to theabove-mentioned configuration, for example, after a target unit regionand a peripheral image thereof are obtained from the frame memory, andstored in the local memory, the smoothing processing by the smoothingmeans and the filter processing by the filter means are consecutivelyperformed on the image stored in the local memory. Therefore, accordingto the above-mentioned configuration, with both of the smoothingprocessing and filter processing, the amount of data to be transferredcan be reduced as compared to the configuration according to the relatedart.

Also, the above-mentioned image filter device further includes referenceregion modifying means configured to reduce a reference region to bereferenced for the filter means calculating a pixel value of each targetpixel to a size according to a position of this target pixel in a unitregion including this target pixel, including first setting meansconfigured to set, when distance between a target pixel and a horizontaledge on the downstream side of a unit region including this target pixelis equal to or shorter than DB, a position of a horizontal edge on thedownstream side of a reference region regarding this target pixel so asto suppress distance with this target pixel to be equal or shorter thanDB, or second setting means configured to set, when distance between atarget pixel and a vertical edge on the downstream side of a unit regionincluding this target pixel is equal to or shorter than DB, a positionof a vertical edge on the downstream side of a reference regionregarding this target pixel so as to suppress distance with this targetpixel to be equal or shorter than DB.

The first setting means set, when distance between a target pixel and ahorizontal edge on the downstream side of a unit region including thistarget pixel is equal to or shorter than DB, a position of a horizontaledge on the downstream side of a reference region regarding this targetpixel so as to suppress distance with this target pixel to be equal orshorter than the DB, and accordingly, the filter means calculate thepixel value of a target pixel without referencing the pixel value of apixel positioned on the downstream side of a horizontal edge on thedownstream side of the unit region. Also, the second setting means set,when distance between a target pixel and a vertical edge on thedownstream side of a unit region including this target pixel is equal toor shorter than the DB, a position of a vertical edge on the downstreamside of a reference region regarding this target pixel so as to suppressdistance with this target pixel to be equal or shorter than the DB, andaccordingly, the filter means calculate the pixel value of a targetpixel without referencing the pixel value of a pixel positioned on thedownstream side of a vertical edge on the downstream side of the unitregion. Accordingly, according to the above-mentioned configuration, theamount of data to be transferred can further be reduced.

Also, it is desirable that the above-mentioned image filter devicefurther include target pixel restricting means configured to restricteach target pixel from which the filter means calculate a pixel value toa pixel where distance from at least one of two edges on the downstreamside of a unit region including this target pixel is equal to or longerthan a predetermined threshold.

According to the above-mentioned configuration, the position of a targetpixel is restricted to a position where distance from at least one oftwo edges on the downstream side of a unit region including this targetpixel is equal to or longer than a predetermined threshold, andaccordingly, the number of pixels that the filter means reference isreduced. Accordingly, according to the above-mentioned configuration,the amount of data to be transferred at the time of filter processingcan be reduced.

Also, an image filter device according to the present inventionincludes: smoothing means configured to smooth a pixel value of a pixelpositioned near at least one of two edges on the upstream side inprocessing order of each unit region regarding a decoded image to beobtained by adding a predicted image to be generated for each unitregion and a residual image; calculating means including first filtermeans configured to act on an output image from the smoothing means,second filter means configured to act on the predicted image, and thirdfilter means configured to act on the residual image, configured to addand output output images from the first to third filter means; andreference region setting means configured to set a reference regionwhich the first to third filter means reference for calculating a pixelvalue of each target pixel, and to set the number of pixels in thevertical direction of a reference region to 1.

According to the above-mentioned image filter device, an output image isgenerated with reference to a decoded image subjected to smoothingprocessing by the smoothing means, a predicted image, and a residualimage. Accordingly, a decoding device including the above-mentionedimage filter device, and an encoding device including theabove-mentioned image filter device generate a predicted image withreference to the output image of the image filter device, whereby codingefficiency improves. Also, the number of pixels in the verticaldirection of the reference region is set to 1, and accordingly, thethroughput of filter processing, and the amount of data to betransferred are reduced.

Also, an image filter device according to the present inventionincludes: filter means configured to generate an output image by actingon an input image made up of one or a plurality of slices; filter meansconfigured to calculate a pixel value of a target pixel in the outputimage from each pixel value of the input image in a reference region setin the vicinity of this target pixel; and target pixel restricting meansconfigured to restrict a position of a target pixel to a position wherethe entire reference region including this target pixel is included in aslice including this target pixel.

According to the above-mentioned image filter device, an output image ina target slice is generated with reference to only a pixel value withinthe target slice. In other words, an output image in a target slice isgenerated without referencing a pixel outside the target slice.Accordingly, according to the above-mentioned configuration, throughputfor generating an output image is reduced. Also, according to theabove-mentioned configuration, filter processing can be performedwithout awaiting until each pixel of the next slice in processing ordergoes into be referable, which is a slice adjacent to the target slice,and accordingly, processing time is reduced.

Also, a decoding device causes the above-mentioned image filter deviceto act on a decoded image, which is a decoding device including theabove-mentioned image filter device, and an encoding device causes theabove-mentioned image filter device to act on a local decoded image,which is an encoding device including the above-mentioned image filterdevice are also included in the category of the present invention. Also,a data structure of encoded data including a filter coefficient group tobe referenced by the above-mentioned image filter is also included inthe category of the present invention.

The present invention is not restricted to the above-mentionedembodiments, various modifications may be made within the scopeindicated in Claims, and an embodiment to be obtained by combiningtechnical means disclosed in different embodiments as appropriate isalso included in the technical range of the present invention.

INDUSTRIAL APPLICABILITY

The present invention may suitably be employed for an image filterdevice configured to perform filtering on images. Also, the presentinvention may suitably be employed for a decoding device configured todecode encoded data, and an encoding device configured to generateencoded data.

REFERENCE SIGNS LIST

-   -   1 moving image decoding device (decoding device)    -   12 predicted image generator    -   15 frame memory    -   16 loop filter (image filter device)    -   161 deblocking filter (smoothing means)    -   162 adaptive filter    -   162 a target pixel setting unit    -   162 b reference region setting unit (reference region modifying        means)    -   162 c filter processing unit (filter means)    -   163 internal memory    -   2 moving image encoding device (encoding device)    -   21 predicted image generator    -   25 frame memory    -   26 loop filter (image filter device)    -   262 adaptive filter    -   262 a target pixel setting unit    -   262 b reference region setting unit (reference region modifying        means)    -   262 c filter processing unit (filter means)

1. (canceled) 2: An image filter device composing: a processor; and amemory associated with the processor, wherein the processor isconfigured to perform steps of: performing deblocking on an image;performing adaptive filter processing for a filter processing unit setto have a same size as with a target largest coding unit (LCU) and tohave a position different from the target LCU, and to be overlapped witha region where deblocking has been performed regarding the target LCU;setting pixels, deblocking has been performed, included in the filterprocessing unit to target pixels in raster scan order; setting areference region of each of the target pixels in a vicinity of the eachof the target pixels based on a position of the each of the targetpixels within the filter processing unit or a position of the each ofthe target pixels within the target LCU, wherein the reference region isdetermined depending on at least one of (1) a number of tap of a filter,(2) width Ddf in a Y direction of a region where deblocking has not beencompleted in the target LCU, (3) length LLCUy in the Y direction of thetarget LCU, and (4) an absolute value Ly of deviation in the Y directionbetween the target LCU and the filter processing unit; calculating apixel value of the each of the target pixels with reference to a pixelvalue of a pixel included in the reference region. 3: The image filterdevice according to the claim 2, wherein the processor restricts anupward reference range Dupper, specifying a distance along a Y axisbetween the target pixel and a pixel adjacent to an upper edge of thereference region, to a number of pixels equal to or smaller than a valueof a Y coordinate of the target pixel within the target LCU in a casethat the Y coordinate of the target pixel within the target LCU is lessthan a predetermined threshold Lupper1. 4: The image filter deviceaccording to the claim 2, wherein the processor restricts an downwardreference range Dlower, specifying a distance along a Y axis between thetarget pixel and a pixel adjacent to a lower edge of the referenceregion, to a number of pixels equal to or smaller than a value ofLLCUy−y−1 in a case that in a Y coordinate y of the target pixel withinthe target LCU is equal to or greater than a predetermined thresholdLLCUy−Llower1. 5: The image filter device according to the claim 2,wherein the processor restricts an upward reference range Dupper,specifying a distance along a Y axis between the target pixel and apixel adjacent to an upper edge of the reference region, to a number ofpixels equal to or smaller than a value of Ly+yr−LLCUy in a case that anormalized Y coordinate yr is equal to or greater than a predeterminedthreshold Lupper2, wherein the normalized Y coordinate yr is a Ycoordinate of a relative coordinate of the target pixel from an upperleft coordinate (0,0) of the target LCU. 6: The image filter deviceaccording to the claim 2, wherein the processor restricts an downwardreference range Dlower, specifying a distance along a Y axis between thetarget pixel and a pixel adjacent to a lower edge of the referenceregion, to a number of pixels equal to or smaller than a value ofLLCUy−Ddf−yr−1 in a case that a normalized Y coordinate yr is less thana predetermined threshold Llower2, wherein the normalized Y coordinateyr is a Y coordinate of a relative coordinate of the target pixel froman upper left coordinate (0,0) of the target LCU. 7: The image filterdevice according to the claim 2, wherein the processor restricts andownward reference range Dlower, specifying a distance along a Y axisbetween the target pixel and a pixel adjacent to a lower edge of thereference region, to a number of pixels equal to or smaller than a valueof LLCUy−Ly−yr−1 in a case that a normalized Y coordinate yr is lessthan a predetermined threshold Llower3, wherein the normalized Ycoordinate yr is a Y coordinate of a relative coordinate of the targetpixel from an upper left coordinate (0,0) of the target LCU.